Audit-driven cleanup pass. End state:
- cargo clippy --release --all-targets: 0 errors (was 4)
- undocumented_unsafe_blocks warnings: 0 (was 67)
- Cargo.toml: undocumented_unsafe_blocks escalated warn -> deny
Clippy correctness errors fixed:
- src/bin/{sw_encode_bench,vaapi_import_bench}.rs: receive_first_frame
rewritten per Oracle plan with total 10s deadline + 200ms wait slice +
while-let drain of all control events. The previous loop body always
exited on first iteration (never_loop); the new version actually retries
and matches production's repeated-poll semantics in state_portal.rs.
- src/avhw.rs: hash_sampled_y_plane tests now use a row_range(row, stride,
width) helper instead of inline stride * N. Preserves the row-index
intent across all sibling tests without tripping erasing_op (row==0) or
identity_op (row==1).
Machine-applicable clippy autofixes applied via 'cargo clippy --fix':
- unnecessary_cast, manual_is_multiple_of, needless_borrows_for_generic_args
- manual_abs_diff, derivable_impls, new_without_default
- unnecessary_map_or, unneeded_struct_pattern, redundant_locals
webrtc_gop_formula test rewritten to wrap the (fps * 2).max(20) formula in
a runtime lambda. The previous clippy --fix pass had constant-folded the
5fps case into assert_eq!(20, 20), silently stripping the floor-case
coverage. The lambda blocks the fold while keeping the formula exercisable.
67 SAFETY comments added across 7 files (cap_portal.rs 26, sw_encode_bench
21, state_portal.rs 7, vaapi_import_bench.rs 6, avhw.rs 5, state.rs 1,
main.rs 1). Two sites carry load-bearing invariant documentation:
- cap_portal.rs:806 process callback documents the PipeWire raw_buf
ownership contract across all 10 exit paths (audited: every path
correctly requeues; fd ownership via dup() is independent and also
exactly-once closed).
- avhw.rs:341 unsafe impl Send for EncState documents the single-thread
exclusivity assumption referenced by AGENTS.md.
All 97 unit tests + 3 integration tests still pass; cargo build --release
finishes clean. Lint escalation to deny freezes the SAFETY baseline: any
future patch adding an unsafe block without a // SAFETY: comment will fail
clippy at compile time.
Final piece of the latency puzzle. Server-side frame_age was 6ms but browser
jitterBufferDelay still spiked to 500+ ms during active periods.
Root cause found via librarian investigation of str0m 0.20 source:
str0m LeakyBucketPacer (active when BWE enabled) limits send rate to
BWE_estimate * 1.1. For a 100KB IDR frame at 8Mbps pacing, the pacer
queue adds ~100ms of send-side latency. Multiple frames stack during
burst encode, causing receiver jitter buffer to grow.
Video streams are PACED BY DEFAULT in str0m (audio is unpaced).
Evidence: str0m/src/streams/send.rs:1116 default unpaced logic.
Fix: call stream_tx.set_unpaced(true) on the video stream when media
is added. BWE remains enabled for bitrate adaptation / TWCC feedback,
but the pacer no longer throttles our video egress.
Why this is the right call for wl-webrtc:
- LAN scenario: dedicated link, no competing flows to smooth against
- Encoder cap (8 Mbps) + VBV buffer (250ms) already provide rate control
- The pacer's smoothing benefit (avoid bursts that compete with TCP) is
irrelevant for our use case
- BWE adaptation still works (we still receive EgressBitrateEstimate events
and adapt encoder bitrate via BitrateCommand::UpdateBitrate)
Verification expectations:
- Active-period jitterBufferDelay: 500+ ms -> < 200 ms
- 'Stop mouse, client continues 10s' symptom: should disappear entirely
- Server-side frame_age: unchanged (~6ms)
- Bitrate/resolution adaptation: unchanged (still uses BWE)
- MP4 mode: unaffected (different code path)
Tests:
- cargo build --release: 0 new warnings (19 baseline preserved)
- cargo test: 96 lib + 3 integration, 0 failed
- SAFETY comments preserved verbatim
- 1 file changed, 7 insertions(+), 1 deletion(-)
Closes the latency investigation started in #23 / #24 / #25.
Root cause:
After #24 fixed PTS propagation, browser jitter buffer still accumulated
to 10+ seconds during active mouse movement. User reported: stop moving
mouse, client continues showing motion for ~10 seconds.
The encoder time_base was 1/fps (33ms granularity at 30fps). When KWin
delivers frames at 60fps (16.7ms apart), compute_capture_pts integer
math mapped multiple captures to the same tick. The monotonicity guard
then bumped them to sequential ticks (0, 1, 2, 3, ...).
Result: 60 captures in 1 real second produced 60 sequential RTP
timestamps spanning 60 * 33ms = 1.98 seconds of RTP time. Browser
played at RTP rate (half real speed), buffer accumulated.
Math verification from test8 log:
- 2067 frames * 3000 RTP jumps = 68s of active RTP time
- 61 frames * 54000 RTP jumps = 37s of static RTP time
- Total RTP time 105s vs real time 98.6s (7% inflation in short session;
long active sessions amplify to 50%+ inflation matching user-reported
10-second trailing).
Fix (per Oracle round review):
Change WebRTC encoder time_base from 1/fps to 1/90000 (90kHz). This
matches the RTP video clock directly, providing 11us PTS granularity.
Captures 16.7ms apart now produce distinct ticks (~1500 each), no
quantization, RTP timestamps accurately reflect real time.
Oracle-required revisions incorporated:
1. **set_frame_rate alongside time_base** — libx264 infers fps from
time_base when not explicit. With 1/90000 time_base and no explicit
framerate, x264 would assume ~90000fps and VBV rate control would
break. Setting framerate=fps/1 preserves real frame semantics while
using 90kHz PTS precision.
2. **rtp_timestamp_from_pts_ticks returns u64 not u32** — MediaTime::new
takes u64. Returning u32 would truncate at 13.25 hours and create
backwards MediaTime. str0m handles RTP u32 wrap internally; we feed
it full u64.
3. **wlr-screencopy path also updated** — state.rs:605 used fps-based
PTS formula. Changed to 90kHz ticks so wlr path matches Portal path
unit. Without this, wlr-screencopy users would have wrong PTS after
the time_base change.
4. **MP4 path (create_software_h264_muxer) UNCHANGED** — verified at
avhw.rs:1693-1789, keeps 1/fps time_base, no set_frame_rate added.
File output doesn't need real-time PTS.
Implementation:
- src/avhw.rs: WEBRTC_RTP_CLOCK_HZ=90_000 const; create_software_h264_encoder
uses 1/90000 time_base + explicit framerate
- src/state_portal.rs: compute_capture_pts uses WEBRTC_RTP_CLOCK_HZ for
tick conversion (was fps multiplier)
- src/state.rs: wlr PTS formula uses 90_000 (was fps multiplier)
- src/webrtc.rs: rtp_timestamp_from_pts_ticks simplified to identity
function (pts_ticks.max(0) as u64), drop fps parameter; write_h264_frame
signature drops fps (was only used for rtp conversion); 5 unit tests
updated to assert 90kHz identity (0->0, 1500->1500, 90000->90000)
Verification expectations:
- Active period jitterBufferDelay: 1000+ ms -> < 100 ms
- 'Stop mouse, client continues 10s' symptom: should disappear
- Static period behavior: unchanged (was already correct)
- MP4 file output: unchanged
- VBV-constrained IDR sizes: unchanged (framerate explicit preserves
rate control semantics)
- All prior fixes (#19, #23, #15, #18, #24) preserved
Out of scope (Oracle noted, not blocking):
- build_swenc_filter_graph still uses 1/fps time_base at avhw.rs:1603/1620
(semantic mismatch but no functional impact since scale_vaapi passes
PTS integers through)
- Runtime VBV update on bitrate change (separate pre-existing issue)
Tests:
- cargo build --release: 0 new warnings (23 baseline preserved)
- cargo test: 96 lib + 3 integration, 0 failed
- 5 rtp_timestamp_* tests updated for 90kHz identity
- SAFETY comments preserved verbatim
- 4 files changed, +48/-35 lines
Root cause (3-stage bug found via Oracle round 1+2 review):
Browser WebRTC clients accumulated 2-3 seconds jitter buffer under
damage-driven variable frame rate (KWin Portal/PipeWire). User moved
mouse, saw action 2-3 seconds later on client.
Three coordinated bugs formed a chain that defeated any single-point fix:
1. state_portal.rs:455 used sequential frame counter as PTS instead of
real capture time. (Portal path only — wlr-screencopy already correct.)
2. avhw.rs Channel output sent only Vec<u8>, DISCARDING AVPacket PTS.
Even with correct encoder PTS, timing metadata was thrown away.
3. webrtc.rs:695 computed RTP timestamp as 'frame_number * 90000 / fps'
from a counter, ignoring any real PTS. Browser saw uniform 33ms RTP
spacing regardless of actual 1.6-57fps variable delivery, growing
jitter buffer to compensate for perceived 'network jitter'.
Fix (7-step ordered implementation per Oracle round 2):
1. EncodedH264Frame struct in avhw.rs carries data + pts_ticks
2. FrameOutput::Channel type changed from Sender<Vec<u8>> to
Sender<EncodedH264Frame>; both new_webrtc signatures updated
3. Channel drain in avhw.rs extracts pkt.pts(), normalizes via
'p - start_ts' (mirrors existing Muxer branch logic). Drops
packets with missing PTS instead of silently emitting zero.
4. write_h264_frame signature: frame_number:u64 -> pts_ticks:i64
(both WebRtcState and WebRtcInner layers). Extracted pure function
rtp_timestamp_from_pts_ticks(pts_ticks, fps) with 5 unit tests
covering zero, one-frame, one-second, negative clamp, fps=0.
5. state_portal.rs receiver loop consumes EncodedH264Frame, passes
.data and .pts_ticks to write_h264_frame.
6. state.rs (wlr-screencopy) receiver loop updated for compile
compatibility — its existing real-time PTS computation at state.rs:606
was already correct, now properly propagates through new channel type.
7. state_portal.rs:467 PTS computation GATED on output mode:
- WebRTC branch: compute_capture_pts() uses PipeWire's ns timestamp
(or Instant fallback), normalizes to first-frame-origin, converts
to encoder time_base units with i128 intermediate math, enforces
monotonicity via safe checked_add pattern.
- MP4 branch: KEEPS self.frames_encoded as i64 (sequential counter).
File output does not need real-time PTS; changing it would alter
file playback speed during static periods.
Oracle round 2 critical revisions incorporated:
- Single-point PTS normalization (only in avhw Channel drain), NOT at
source. Avoids double-subtraction with existing Muxer logic.
- MP4 path explicitly preserved — real PTS only applied to WebRTC branch.
- Safe Rust monotonicity guard (no unsafe pointer tricks).
- i64::try_from(ticks_i128) instead of broken i128::try_from(...).unwrap_or(i64::MAX).
- pkt.pts() missing -> log + drop, not silent unwrap_or(0).
- Updates span 4 files (avhw.rs, webrtc.rs, state_portal.rs, state.rs)
because channel type change ripples through both Portal and wlr paths.
Verification expectations:
- Browser jitter buffer should stabilize at 100-500ms (typical) instead
of growing to 2-3 seconds under damage-driven delivery.
- chrome://webrtc-internals: jitterBufferDelay / jitterBufferEmittedCount
ratio should drop significantly.
- Server-side metrics (output_bps, frame rate, IDR size) unchanged.
- MP4 file output (--output mode) behavior unchanged.
Out of scope (deferred):
- frame_age metric fix (Oracle: separate commit to isolate behavioral
change from observability change)
- VFR encoder redesign (1/fps time_base sufficient for this fix)
- MP4 VFR recording (semantic change, separate decision)
- Existing client jitter buffers may not auto-shrink; reconnect may be
required for users with already-accumulated latency
Tests:
- cargo build --release: clean, 0 new warnings (19 pre-existing)
- cargo test: 96 lib + 96 bin + 3 integration, 0 failed
- 5 new RTP unit tests covering edge cases
- SAFETY comments preserved verbatim
- 4 files changed, +157/-27 lines
Root causes (3, discovered via Oracle review + x264 source verification):
A. PLI storm: WebRtcInner.force_keyframe_to_encode is a single bool with
no rate limit. Client PLI storms (5 PLIs in 845ms observed) produced
5 back-to-back IDRs (~1.5MB burst), swamping the network.
B. Bitrate runaway: BWE feedback had no upper bound. Observed bitrate
escalating from 5 Mbps to 9.9 Mbps in 12 seconds, all applied to
encoder. Combined with A, made each IDR grow 4-5x.
C. VBV effectively disabled (NEW finding not in original issue):
x264's vbv-maxrate and vbv-bufsize x264opts expect kbit/s and kbit
(confirmed via x264 source ratecontrol.c:658-661 which multiplies by
1000 at use site). The code passed bps, making VBV interpret 5.5 Mbps
as 5.5 Gbps (clipped to 2 Gbps). Buffer was 173MB instead of 172KB,
so VBV never constrained anything. This is why IDRs could balloon to
336KB after runtime bitrate increases.
Fixes (3, all in this commit per Oracle review):
Fix 0 (avhw.rs): divide bitrate by 1000 when formatting x264opts string.
Updated existing vbv_x264opts_format and vbv_bufsize_is_quarter_of_maxrate
tests to assert correct kbit/s values. Old tests passed but asserted
wrong values - classic 'tests covered the wrong implementation'.
Fix 1 (webrtc.rs): split keyframe trigger into two paths.
- set_need_keyframe() (internal: connect, resolution change) remains
unthrottled but updates last_forced_keyframe_at timestamp.
- request_keyframe_from_viewer() (external PLI) rate-limited to
Duration::from_secs(1), checked against last_forced_keyframe_at.
- Key insight from Oracle: track ALL keyframe production time, not
just PLI time, to prevent 'connect -> immediate PLI -> duplicate IDR'.
Fix 2 (args.rs + state_portal.rs + avhw.rs):
- New --max-bitrate CLI flag, default 8 Mbps.
- Primary clamp in webrtc_thread_loop (policy layer): clamp BWE via
variable shadowing so all downstream code (bitrate_tx, resolution
adaptation) uses clamped value.
- Defensive guardrail in encode_cpu_frame UpdateBitrate handler:
50 Mbps hard ceiling in case future callers bypass policy layer.
- Per Oracle: flat 8 Mbps default, no auto-scaling
(max(8M, 2*initial) would have failed the observed case).
Verification (82.1s session, 34.7 fps avg vs previous 18.2):
PLI storm absorption:
- 15 PLIs received from viewer
- 10 PLIs throttled (67%)
- 6 IDRs produced total (1 connect + 5 honored)
- 3 distinct PLI storms (625ms, 640ms, 858ms duration) all absorbed
VBV constraint working:
- First IDR: 65KB (was 67KB)
- Largest IDR: 169KB (was 336KB)
- All IDRs under VBV buffer bound of 172KB
- No more runaway IDR growth
Bitrate cap working:
- 5 bitrate updates applied (was 8)
- Peak bitrate: 7.4 Mbps (was 9.9 Mbps)
- 52952 BWE readings clamped (>8 Mbps filtered out)
Overall quality:
- Frame rate: 34.7 fps (was 18.2, 1.9x improvement, exceeds 30 fps target)
- Average bitrate: 4451 kb/s (was 5616, lower and more stable)
Out of scope (deferred to future work):
- Runtime VBV reconfiguration on bitrate change (Fix 3): encoder
recreation is expensive, observe whether VBV mismatch becomes a
quality issue after cap is in place
- Asymmetric BWE filter (Fix 4): rise=15%/fall=5% thresholds; nice
tuning but cap is sufficient for now
- Compositor stalls (#15): still causes some stutter at session end
but no longer compounds into latency
Tests:
- cargo test: 91 passed + 3 passed + 0 failed
- vbv_x264opts_format and vbv_bufsize_is_quarter_of_maxrate updated
and passing with new kbit/s assertions
- SAFETY comments preserved verbatim
- cargo build --release: clean, no new warnings
WebRTC client bandwidth estimate now drives both encoder bitrate and
resolution tier selection, replacing the previous static-target encoder.
- webrtc.rs: enable str0m BWE (seeded at 5 Mbps), surface
EgressBitrateEstimate + KeyframeRequest events, expose
get_bwe_estimate() / set_need_keyframe()
- state_portal.rs: wire bitrate/resolution channels between the WebRTC
thread and the encode thread; tier ladder [1440p, 1080p, 720p] with
downscale at 60% budget and upscale hysteresis (120% sustained 10s)
- avhw.rs: SwEncImport::poll_resolution_commands() rebuilds the import
filter graph on UpdateResolution; SwEncEncode::recreate_encoder()
rebuilds sws/enc_video/yuv_frame atomically; hash_sampled_y_plane()
skips duplicate frames; VBV x264opts cap IDR bursts; H.264 level 4.0
(muxer) / 4.2 (WebRTC)
- state.rs: sync wlr-screencopy GOP to fps*2 max 20 for parity
- fix: drain bitrate_rx + resolution_rx BEFORE the stride check in
encode_cpu_frame() so the new (smaller-stride) frame produced after
a resolution change does not hit the stale (larger) enc_width and
crash the encode thread
- WebRTC GOP widened to fps*2 max 20 (was fps/2 max 10)
The 500 error response previously included the raw error message {e}
in the body, potentially leaking internal implementation details (SDP
parse errors, ICE candidate info) to clients.
The detailed error is already logged server-side via tracing::error!,
so the response body is now a fixed generic string with a proper
HTTP/1.1 status line.
poll_rtc() always returned Ok(false), preventing WebRtcState from
clearing self.inner on disconnect. This leaked the UDP socket, Rtc
instance, and 65KB buffer permanently if the client never reconnected.
Closes#10
- Add src/webrtc.rs: HTTP signaling server + str0m Sans-IO WebRTC transport
with H.264 Annex-B → RTP packetization and key-frame request handling
- avhw: introduce FrameOutput enum (Muxer | Channel) so SwEncState can
output to either MP4 muxer or crossbeam channel for WebRTC
- cap_portal: support portal session restore tokens (PersistMode::ExplicitlyRevoked)
to skip re-authorization dialog; add --no-persist flag to force fresh dialog
- args: make --output optional when --port is used for WebRTC mode
- state_portal: integrate WebRTC pipeline (encoder channel → RTP forwarding)
with shorter GOP for WebRTC (fps/2, min 10)
- main: redirect tracing to stderr; validate --output or --port required
- Add dependencies: str0m 0.20, serde_json 1, dirs 6
dailz