Commit Graph
7 Commits
Author SHA1 Message Date
dailz 9a522e2f99 fix(webrtc): PLI rate limiting + bitrate cap + VBV units (closes #23)
Root causes (3, discovered via Oracle review + x264 source verification):

A. PLI storm: WebRtcInner.force_keyframe_to_encode is a single bool with
   no rate limit. Client PLI storms (5 PLIs in 845ms observed) produced
   5 back-to-back IDRs (~1.5MB burst), swamping the network.

B. Bitrate runaway: BWE feedback had no upper bound. Observed bitrate
   escalating from 5 Mbps to 9.9 Mbps in 12 seconds, all applied to
   encoder. Combined with A, made each IDR grow 4-5x.

C. VBV effectively disabled (NEW finding not in original issue):
   x264's vbv-maxrate and vbv-bufsize x264opts expect kbit/s and kbit
   (confirmed via x264 source ratecontrol.c:658-661 which multiplies by
   1000 at use site). The code passed bps, making VBV interpret 5.5 Mbps
   as 5.5 Gbps (clipped to 2 Gbps). Buffer was 173MB instead of 172KB,
   so VBV never constrained anything. This is why IDRs could balloon to
   336KB after runtime bitrate increases.

Fixes (3, all in this commit per Oracle review):

Fix 0 (avhw.rs): divide bitrate by 1000 when formatting x264opts string.
  Updated existing vbv_x264opts_format and vbv_bufsize_is_quarter_of_maxrate
  tests to assert correct kbit/s values. Old tests passed but asserted
  wrong values - classic 'tests covered the wrong implementation'.

Fix 1 (webrtc.rs): split keyframe trigger into two paths.
  - set_need_keyframe() (internal: connect, resolution change) remains
    unthrottled but updates last_forced_keyframe_at timestamp.
  - request_keyframe_from_viewer() (external PLI) rate-limited to
    Duration::from_secs(1), checked against last_forced_keyframe_at.
  - Key insight from Oracle: track ALL keyframe production time, not
    just PLI time, to prevent 'connect -> immediate PLI -> duplicate IDR'.

Fix 2 (args.rs + state_portal.rs + avhw.rs):
  - New --max-bitrate CLI flag, default 8 Mbps.
  - Primary clamp in webrtc_thread_loop (policy layer): clamp BWE via
    variable shadowing so all downstream code (bitrate_tx, resolution
    adaptation) uses clamped value.
  - Defensive guardrail in encode_cpu_frame UpdateBitrate handler:
    50 Mbps hard ceiling in case future callers bypass policy layer.
  - Per Oracle: flat 8 Mbps default, no auto-scaling
    (max(8M, 2*initial) would have failed the observed case).

Verification (82.1s session, 34.7 fps avg vs previous 18.2):

PLI storm absorption:
  - 15 PLIs received from viewer
  - 10 PLIs throttled (67%)
  - 6 IDRs produced total (1 connect + 5 honored)
  - 3 distinct PLI storms (625ms, 640ms, 858ms duration) all absorbed

VBV constraint working:
  - First IDR: 65KB (was 67KB)
  - Largest IDR: 169KB (was 336KB)
  - All IDRs under VBV buffer bound of 172KB
  - No more runaway IDR growth

Bitrate cap working:
  - 5 bitrate updates applied (was 8)
  - Peak bitrate: 7.4 Mbps (was 9.9 Mbps)
  - 52952 BWE readings clamped (>8 Mbps filtered out)

Overall quality:
  - Frame rate: 34.7 fps (was 18.2, 1.9x improvement, exceeds 30 fps target)
  - Average bitrate: 4451 kb/s (was 5616, lower and more stable)

Out of scope (deferred to future work):
  - Runtime VBV reconfiguration on bitrate change (Fix 3): encoder
    recreation is expensive, observe whether VBV mismatch becomes a
    quality issue after cap is in place
  - Asymmetric BWE filter (Fix 4): rise=15%/fall=5% thresholds; nice
    tuning but cap is sufficient for now
  - Compositor stalls (#15): still causes some stutter at session end
    but no longer compounds into latency

Tests:
  - cargo test: 91 passed + 3 passed + 0 failed
  - vbv_x264opts_format and vbv_bufsize_is_quarter_of_maxrate updated
    and passing with new kbit/s assertions
  - SAFETY comments preserved verbatim
  - cargo build --release: clean, no new warnings
2026-06-20 20:36:44 +08:00
dailz 826f544569 feat(portal): async encode pipeline - decouple capture from encoding
Split synchronous encode pipeline so sws_scale + libx264 runs on a
dedicated thread, leaving only VAAPI import + GPU scale + GPU→CPU
transfer on the main capture thread.

Problem: encode_p95 occasionally hit 74ms, blocking the entire capture
pipeline and causing capture_gap_max=356ms stutter.

Solution:
- avhw.rs: Split SwEncState into SwEncImport (main thread: VAAPI import,
  filter_graph scale, GPU→CPU transfer) and SwEncEncode (encode thread:
  sws_scale NV12→YUV420P, libx264 encode). New CpuNv12Frame struct
  carries owned pixel data across threads via crossbeam channel.
  SwEncState wraps both for backward compat (MP4/sync path untouched).
- state_portal.rs: WebRTC portal path spawns 'wl-webrtc-encode' thread
  with bounded(2) input channel (drop-newest backpressure) and separate
  timing channel. Graceful shutdown: drop webrtc_rx → drop input_tx →
  join encode thread → flush sync encoder.
- stats.rs: Add record_import() + record_encode_thread() for async timing.

Results: encode_p95 stable at 2.9-4.2ms (was 11-74ms), capture_fps
stable 59-60fps, cap_gap_p95 17-19ms. Remaining capture stalls traced
to PipeWire compositor frame delivery (external, not our code).
2026-06-07 16:55:28 +08:00
dailz 46367ef6b5 fix(state): add WebRTC support to wlr-screencopy backend
Fixes #1 -- --port mode with wlr-screencopy backend caused panic at
negotiate_format() because self.args.output is None and .expect() was
called unconditionally.

Changes:
- Introduce StreamingEncoder enum wrapping EncState (MP4) and
  SwEncState (WebRTC) with unified frames_rgb/encode_frame/flush API
- Add WebRTC fields to State<S> (webrtc, webrtc_tx, webrtc_rx,
  webrtc_frames_sent) matching Portal backend pattern
- State::new() returns Result<Self> for clean WebRtcState init failure
- negotiate_format() branches on webrtc_tx: WebRTC path uses
  SwEncState::new_webrtc(), MP4 path unchanged (hardware VAAPI)
- Add poll_webrtc() method to drive signaling + channel drain
- Event loop calls poll_webrtc() each iteration
- Fix pre-existing test/bench Args construction (Option<String> output,
  missing no_persist field)
2026-06-04 22:10:46 +08:00
dailz d80b34f44f feat: GPU-downscale + software H.264 encode pipeline (WIP)
Add SwEncState in avhw.rs: GPU pipeline using scale_vaapi to downscale
4K BGRA -> 2K NV12 on AMD iGPU, then software encode with libopenh264.

- import_dma_buf_to_vaapi: av_hwframe_map based DMA-BUF import
- SwEncState: GPU filter graph (scale_vaapi) + NV12->YUV420P + libopenh264
- state_portal.rs: integrated SwEncState, auto DRM device detection
- vaapi_import_bench.rs: CPU vs GPU pipeline benchmark
- sw_encode_bench.rs: software encode benchmark

Benchmark results: GPU pipeline ~91 FPS theoretical (10.95ms/frame)
vs CPU pipeline ~33 FPS (30.21ms/frame).

Known issue: only 1 frame encoded in production recording,
diagnostic STATS logging added to debug frame flow.
2026-05-29 22:04:12 +08:00
dailz 55abb5e56d fix(backend_detect): use raw zbus for portal check to avoid OnceLock connection poisoning
ashpd caches zbus::Connection in a global OnceLock. When check_portal_available()
created a Screencast proxy, the connection was cached there. When the function
returned and its tokio Runtime dropped, the cached connection became dead.
Subsequent setup_portal() calls reused this dead connection and hung forever.

Fix: replace ashpd Screencast proxy with direct zbus D-Bus interface check,
which does not touch the ashpd global connection cache.

Add examples/test_portal.rs for minimal Portal ScreenCast testing.
2026-05-27 22:07:11 +08:00
dailz 25110e8463 feat(backend_detect): add portal/screencopy availability pre-checks
Add check_portal_available() and check_screencopy_available() to
probe each backend independently before committing. This enables
smarter fallback logic and better diagnostics when no backend is
found. Includes Chinese documentation comments.
2026-05-25 08:56:33 +08:00
dailz d7fbb5256c feat: add KWin/KDE Plasma screen capture via xdg-desktop-portal ScreenCast + PipeWire
Add a second capture backend for compositors without wlr-screencopy
(KWin, GNOME, etc.) using the xdg-desktop-portal ScreenCast interface
and PipeWire DMA-BUF streaming.

New files:
- src/backend_detect.rs: auto-detect wlr-screencopy vs portal backend
- src/cap_portal.rs: Portal session setup + PipeWire DMA-BUF thread
- src/state_portal.rs: StatePortal encoder pipeline (DMA-BUF → VAAPI)

Changes:
- Cargo.toml: add ashpd 0.13, tokio 1, pipewire 0.9, libspa 0.9,
  crossbeam-channel 0.5
- src/args.rs: add --backend CLI flag
- src/avhw.rs: extract create_encoder() from inline State code
- src/main.rs: route to portal or wlr-screencopy based on backend
- src/state.rs: fix params.destroy() on dup failure, cleanup
  in_flight_surface on copy fail, use create_encoder()
- tests/integration_test.rs: add --backend flag tests
2026-05-11 08:49:08 +08:00