fix(webrtc): switch encoder time_base to 90kHz to stop RTP time inflation (closes #25)
Root cause: After #24 fixed PTS propagation, browser jitter buffer still accumulated to 10+ seconds during active mouse movement. User reported: stop moving mouse, client continues showing motion for ~10 seconds. The encoder time_base was 1/fps (33ms granularity at 30fps). When KWin delivers frames at 60fps (16.7ms apart), compute_capture_pts integer math mapped multiple captures to the same tick. The monotonicity guard then bumped them to sequential ticks (0, 1, 2, 3, ...). Result: 60 captures in 1 real second produced 60 sequential RTP timestamps spanning 60 * 33ms = 1.98 seconds of RTP time. Browser played at RTP rate (half real speed), buffer accumulated. Math verification from test8 log: - 2067 frames * 3000 RTP jumps = 68s of active RTP time - 61 frames * 54000 RTP jumps = 37s of static RTP time - Total RTP time 105s vs real time 98.6s (7% inflation in short session; long active sessions amplify to 50%+ inflation matching user-reported 10-second trailing). Fix (per Oracle round review): Change WebRTC encoder time_base from 1/fps to 1/90000 (90kHz). This matches the RTP video clock directly, providing 11us PTS granularity. Captures 16.7ms apart now produce distinct ticks (~1500 each), no quantization, RTP timestamps accurately reflect real time. Oracle-required revisions incorporated: 1. **set_frame_rate alongside time_base** — libx264 infers fps from time_base when not explicit. With 1/90000 time_base and no explicit framerate, x264 would assume ~90000fps and VBV rate control would break. Setting framerate=fps/1 preserves real frame semantics while using 90kHz PTS precision. 2. **rtp_timestamp_from_pts_ticks returns u64 not u32** — MediaTime::new takes u64. Returning u32 would truncate at 13.25 hours and create backwards MediaTime. str0m handles RTP u32 wrap internally; we feed it full u64. 3. **wlr-screencopy path also updated** — state.rs:605 used fps-based PTS formula. Changed to 90kHz ticks so wlr path matches Portal path unit. Without this, wlr-screencopy users would have wrong PTS after the time_base change. 4. **MP4 path (create_software_h264_muxer) UNCHANGED** — verified at avhw.rs:1693-1789, keeps 1/fps time_base, no set_frame_rate added. File output doesn't need real-time PTS. Implementation: - src/avhw.rs: WEBRTC_RTP_CLOCK_HZ=90_000 const; create_software_h264_encoder uses 1/90000 time_base + explicit framerate - src/state_portal.rs: compute_capture_pts uses WEBRTC_RTP_CLOCK_HZ for tick conversion (was fps multiplier) - src/state.rs: wlr PTS formula uses 90_000 (was fps multiplier) - src/webrtc.rs: rtp_timestamp_from_pts_ticks simplified to identity function (pts_ticks.max(0) as u64), drop fps parameter; write_h264_frame signature drops fps (was only used for rtp conversion); 5 unit tests updated to assert 90kHz identity (0->0, 1500->1500, 90000->90000) Verification expectations: - Active period jitterBufferDelay: 1000+ ms -> < 100 ms - 'Stop mouse, client continues 10s' symptom: should disappear - Static period behavior: unchanged (was already correct) - MP4 file output: unchanged - VBV-constrained IDR sizes: unchanged (framerate explicit preserves rate control semantics) - All prior fixes (#19, #23, #15, #18, #24) preserved Out of scope (Oracle noted, not blocking): - build_swenc_filter_graph still uses 1/fps time_base at avhw.rs:1603/1620 (semantic mismatch but no functional impact since scale_vaapi passes PTS integers through) - Runtime VBV update on bitrate change (separate pre-existing issue) Tests: - cargo build --release: 0 new warnings (23 baseline preserved) - cargo test: 96 lib + 3 integration, 0 failed - 5 rtp_timestamp_* tests updated for 90kHz identity - SAFETY comments preserved verbatim - 4 files changed, +48/-35 lines
This commit is contained in:
+5
-4
@@ -553,7 +553,7 @@ impl StatePortal {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Compute PTS in encoder time_base units (1/fps) from PipeWire's nanosecond
|
||||
/// Compute PTS in 90kHz media-clock ticks from PipeWire's nanosecond
|
||||
/// capture timestamp. Falls back to `Instant`-based elapsed time when PipeWire
|
||||
/// does not provide PTS. Maintains strict monotonicity (encoder requirement).
|
||||
fn compute_capture_pts(&mut self, pw_pts_ns: i64) -> i64 {
|
||||
@@ -578,7 +578,8 @@ impl StatePortal {
|
||||
self.first_pts_ns = Some(raw_ns);
|
||||
0
|
||||
};
|
||||
let ticks_i128 = (relative_ns.saturating_mul(i128::from(self.args.fps))) / NS_PER_SEC;
|
||||
let ticks_i128 =
|
||||
(relative_ns.saturating_mul(crate::avhw::WEBRTC_RTP_CLOCK_HZ)) / NS_PER_SEC;
|
||||
let computed_pts = i64::try_from(ticks_i128).unwrap_or(i64::MAX);
|
||||
let mut pts = computed_pts;
|
||||
|
||||
@@ -791,7 +792,7 @@ fn webrtc_thread_loop(
|
||||
|
||||
if connected {
|
||||
while let Ok(enc_frame) = webrtc_rx.try_recv() {
|
||||
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks, fps) {
|
||||
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks) {
|
||||
tracing::debug!("WebRTC write frame error: {e}");
|
||||
}
|
||||
frames_sent = frames_sent.saturating_add(1);
|
||||
@@ -808,7 +809,7 @@ fn webrtc_thread_loop(
|
||||
match webrtc_rx.recv_timeout(timeout) {
|
||||
Ok(enc_frame) => {
|
||||
if wrtc.is_connected() {
|
||||
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks, fps)
|
||||
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks)
|
||||
{
|
||||
tracing::debug!("WebRTC write frame error: {e}");
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user