Commit Graph
49 Commits
Author SHA1 Message Date
dailz 079611acfc fix(webrtc): propagate real capture PTS through WebRTC channel (closes #24)
Root cause (3-stage bug found via Oracle round 1+2 review):

Browser WebRTC clients accumulated 2-3 seconds jitter buffer under
damage-driven variable frame rate (KWin Portal/PipeWire). User moved
mouse, saw action 2-3 seconds later on client.

Three coordinated bugs formed a chain that defeated any single-point fix:

1. state_portal.rs:455 used sequential frame counter as PTS instead of
   real capture time. (Portal path only — wlr-screencopy already correct.)

2. avhw.rs Channel output sent only Vec<u8>, DISCARDING AVPacket PTS.
   Even with correct encoder PTS, timing metadata was thrown away.

3. webrtc.rs:695 computed RTP timestamp as 'frame_number * 90000 / fps'
   from a counter, ignoring any real PTS. Browser saw uniform 33ms RTP
   spacing regardless of actual 1.6-57fps variable delivery, growing
   jitter buffer to compensate for perceived 'network jitter'.

Fix (7-step ordered implementation per Oracle round 2):

1. EncodedH264Frame struct in avhw.rs carries data + pts_ticks
2. FrameOutput::Channel type changed from Sender<Vec<u8>> to
   Sender<EncodedH264Frame>; both new_webrtc signatures updated
3. Channel drain in avhw.rs extracts pkt.pts(), normalizes via
   'p - start_ts' (mirrors existing Muxer branch logic). Drops
   packets with missing PTS instead of silently emitting zero.
4. write_h264_frame signature: frame_number:u64 -> pts_ticks:i64
   (both WebRtcState and WebRtcInner layers). Extracted pure function
   rtp_timestamp_from_pts_ticks(pts_ticks, fps) with 5 unit tests
   covering zero, one-frame, one-second, negative clamp, fps=0.
5. state_portal.rs receiver loop consumes EncodedH264Frame, passes
   .data and .pts_ticks to write_h264_frame.
6. state.rs (wlr-screencopy) receiver loop updated for compile
   compatibility — its existing real-time PTS computation at state.rs:606
   was already correct, now properly propagates through new channel type.
7. state_portal.rs:467 PTS computation GATED on output mode:
   - WebRTC branch: compute_capture_pts() uses PipeWire's ns timestamp
     (or Instant fallback), normalizes to first-frame-origin, converts
     to encoder time_base units with i128 intermediate math, enforces
     monotonicity via safe checked_add pattern.
   - MP4 branch: KEEPS self.frames_encoded as i64 (sequential counter).
     File output does not need real-time PTS; changing it would alter
     file playback speed during static periods.

Oracle round 2 critical revisions incorporated:

- Single-point PTS normalization (only in avhw Channel drain), NOT at
  source. Avoids double-subtraction with existing Muxer logic.
- MP4 path explicitly preserved — real PTS only applied to WebRTC branch.
- Safe Rust monotonicity guard (no unsafe pointer tricks).
- i64::try_from(ticks_i128) instead of broken i128::try_from(...).unwrap_or(i64::MAX).
- pkt.pts() missing -> log + drop, not silent unwrap_or(0).
- Updates span 4 files (avhw.rs, webrtc.rs, state_portal.rs, state.rs)
  because channel type change ripples through both Portal and wlr paths.

Verification expectations:

- Browser jitter buffer should stabilize at 100-500ms (typical) instead
  of growing to 2-3 seconds under damage-driven delivery.
- chrome://webrtc-internals: jitterBufferDelay / jitterBufferEmittedCount
  ratio should drop significantly.
- Server-side metrics (output_bps, frame rate, IDR size) unchanged.
- MP4 file output (--output mode) behavior unchanged.

Out of scope (deferred):

- frame_age metric fix (Oracle: separate commit to isolate behavioral
  change from observability change)
- VFR encoder redesign (1/fps time_base sufficient for this fix)
- MP4 VFR recording (semantic change, separate decision)
- Existing client jitter buffers may not auto-shrink; reconnect may be
  required for users with already-accumulated latency

Tests:
- cargo build --release: clean, 0 new warnings (19 pre-existing)
- cargo test: 96 lib + 96 bin + 3 integration, 0 failed
- 5 new RTP unit tests covering edge cases
- SAFETY comments preserved verbatim
- 4 files changed, +157/-27 lines
2026-06-20 21:53:27 +08:00
dailz 2f0b858920 fix(state_portal): remove filler + accept Wayland damage-driven delivery (fixes #15, fixes #18)
Paradigm shift: stop treating KWin's damage-driven frame delivery as an
anomaly. Static content = no new frames is correct Wayland behavior.

Root causes (combined #15 + #18):

#15: stall detection threshold was 100ms (max(100ms, 3*frame_interval)).
KWin/PipeWire damage-driven delivery meant normal static periods triggered
WARN 'compositor frame delivery stalled' continuously. Test3 data showed
55% stall rate during active streaming (38 stalls in 69s). User perceived
severe stutter pattern 'cardboard-effect freeze-release-freeze'.

#18: filler mechanism (maybe_send_filler_frame) cloned 3MB NV12 data and
sent to encode thread during stalls. Encode thread hashed Y plane, found
duplicate, skipped via dedup. Net: wasted CPU + channel bandwidth with
zero visual benefit (the dedup path was already catching it).

Oracle review revealed the two issues are causally linked: filler is the
failed response to the false stall alarm. Removing both together is correct.

Fixes (all in state_portal.rs):

1. Remove filler mechanism entirely:
   - Remove fields: last_fillable_frame, next_filler_at, filler_frames_sent
   - Remove method: maybe_send_filler_frame (49 lines)
   - Remove constant: MAX_FILLER_DURATION
   - Remove fillable_frame clone cascade in handle_pw_frame (10 lines
     of 3MB NV12 cloning per frame, the largest CPU/memory win)
   - Remove filler_frames_sent from stats output

2. Redefine stall as idle (Oracle-revised):
   - Rename: stall_start -> idle_log_start (semantic clarity)
   - Change threshold: 100ms -> 5s (CAPTURE_IDLE_LOG_THRESHOLD)
   - Change log level: WARN -> DEBUG
   - Change wording: 'compositor frame delivery stalled' ->
     'portal capture idle; no damage frames received (normal Wayland behavior)'
   - One-shot log per idle episode (not repeated every second)
   - Use last_capture_arrival as idle start for accurate elapsed duration
     (old code set stall_start=now at first detection, undercounting by
     threshold value)

Explicit product decision (Oracle flagged trade-off):

  Static-content PLI repair is deferred. When WebRTC client sends PLI
  during static content:
  - Server sets force_keyframe_pending in encode thread
  - Encode thread blocks on input_rx.recv() (no frames coming)
  - Client may send more PLIs (all rate-limited by #23 to 1/sec)
  - When user interacts -> KWin delivers frame -> encode thread produces IDR

  This means during fully static content, client may wait for next damage
  to receive keyframe. Acceptable because static content is by definition
  unchanged - the last received frame is still visually accurate. If user
  reports unacceptable PLI latency on static screens, follow-up with
  event-driven one-shot IDR mechanism (Option B per Oracle).

Verification expectation:
  - Zero WARN 'stalled' messages during normal session
  - Optional DEBUG 'portal capture idle' after 5s of no frames
  - Optional DEBUG 'portal capture resumed after idle period' on recovery
  - capture_fps will still vary with content activity (this is correct)
  - encoded_fps will only count real frames (filler no longer inflates it)

Out of scope:
  - Event-driven cached-frame IDR (Option B): follow-up if needed
  - PipeWire CursorFromCache negotiation: separate enhancement
  - capture_fps expectations documentation: defer to #20 stats rework

Tests:
  - cargo build --release: clean, no new warnings
  - cargo test: 91 passed + 3 passed + 0 failed
  - SAFETY comments preserved verbatim
  - Net change: -80 lines (20 insertions, 100 deletions)

Closes #15, closes #18.
2026-06-20 20:56:34 +08:00
dailz 9a522e2f99 fix(webrtc): PLI rate limiting + bitrate cap + VBV units (closes #23)
Root causes (3, discovered via Oracle review + x264 source verification):

A. PLI storm: WebRtcInner.force_keyframe_to_encode is a single bool with
   no rate limit. Client PLI storms (5 PLIs in 845ms observed) produced
   5 back-to-back IDRs (~1.5MB burst), swamping the network.

B. Bitrate runaway: BWE feedback had no upper bound. Observed bitrate
   escalating from 5 Mbps to 9.9 Mbps in 12 seconds, all applied to
   encoder. Combined with A, made each IDR grow 4-5x.

C. VBV effectively disabled (NEW finding not in original issue):
   x264's vbv-maxrate and vbv-bufsize x264opts expect kbit/s and kbit
   (confirmed via x264 source ratecontrol.c:658-661 which multiplies by
   1000 at use site). The code passed bps, making VBV interpret 5.5 Mbps
   as 5.5 Gbps (clipped to 2 Gbps). Buffer was 173MB instead of 172KB,
   so VBV never constrained anything. This is why IDRs could balloon to
   336KB after runtime bitrate increases.

Fixes (3, all in this commit per Oracle review):

Fix 0 (avhw.rs): divide bitrate by 1000 when formatting x264opts string.
  Updated existing vbv_x264opts_format and vbv_bufsize_is_quarter_of_maxrate
  tests to assert correct kbit/s values. Old tests passed but asserted
  wrong values - classic 'tests covered the wrong implementation'.

Fix 1 (webrtc.rs): split keyframe trigger into two paths.
  - set_need_keyframe() (internal: connect, resolution change) remains
    unthrottled but updates last_forced_keyframe_at timestamp.
  - request_keyframe_from_viewer() (external PLI) rate-limited to
    Duration::from_secs(1), checked against last_forced_keyframe_at.
  - Key insight from Oracle: track ALL keyframe production time, not
    just PLI time, to prevent 'connect -> immediate PLI -> duplicate IDR'.

Fix 2 (args.rs + state_portal.rs + avhw.rs):
  - New --max-bitrate CLI flag, default 8 Mbps.
  - Primary clamp in webrtc_thread_loop (policy layer): clamp BWE via
    variable shadowing so all downstream code (bitrate_tx, resolution
    adaptation) uses clamped value.
  - Defensive guardrail in encode_cpu_frame UpdateBitrate handler:
    50 Mbps hard ceiling in case future callers bypass policy layer.
  - Per Oracle: flat 8 Mbps default, no auto-scaling
    (max(8M, 2*initial) would have failed the observed case).

Verification (82.1s session, 34.7 fps avg vs previous 18.2):

PLI storm absorption:
  - 15 PLIs received from viewer
  - 10 PLIs throttled (67%)
  - 6 IDRs produced total (1 connect + 5 honored)
  - 3 distinct PLI storms (625ms, 640ms, 858ms duration) all absorbed

VBV constraint working:
  - First IDR: 65KB (was 67KB)
  - Largest IDR: 169KB (was 336KB)
  - All IDRs under VBV buffer bound of 172KB
  - No more runaway IDR growth

Bitrate cap working:
  - 5 bitrate updates applied (was 8)
  - Peak bitrate: 7.4 Mbps (was 9.9 Mbps)
  - 52952 BWE readings clamped (>8 Mbps filtered out)

Overall quality:
  - Frame rate: 34.7 fps (was 18.2, 1.9x improvement, exceeds 30 fps target)
  - Average bitrate: 4451 kb/s (was 5616, lower and more stable)

Out of scope (deferred to future work):
  - Runtime VBV reconfiguration on bitrate change (Fix 3): encoder
    recreation is expensive, observe whether VBV mismatch becomes a
    quality issue after cap is in place
  - Asymmetric BWE filter (Fix 4): rise=15%/fall=5% thresholds; nice
    tuning but cap is sufficient for now
  - Compositor stalls (#15): still causes some stutter at session end
    but no longer compounds into latency

Tests:
  - cargo test: 91 passed + 3 passed + 0 failed
  - vbv_x264opts_format and vbv_bufsize_is_quarter_of_maxrate updated
    and passing with new kbit/s assertions
  - SAFETY comments preserved verbatim
  - cargo build --release: clean, no new warnings
2026-06-20 20:36:44 +08:00
dailz f38adf70f9 fix(state_portal): gate handle_pw_frame on WebRTC paused state (closes #19)
Root cause (verified via code reading + Oracle design review):
- webrtc_paused IS correctly initialized to true in StatePortal::new()
- encode_cpu_frame DOES respect paused via early return
- BUT encode_thread_loop unconditionally sent timing on every Ok(()),
  causing stats.record_encode_thread to tick encoded_frames even for
  paused-dropped frames -> phantom encoded_fps=29.7 during idle
- Real waste: handle_pw_frame imports DMA-BUF + VAAPI scale + NV12 clone
  + crossbeam send at 60fps even when no WebRTC client is connected

Fix (Option B - surgical bugfix):
- Add EncodeOutcome enum (Encoded/SkippedPaused/SkippedDisconnected/
  SkippedDuplicate) to encode_cpu_frame return type
- encode_thread_loop only reports timing on Ok(Encoded), not on skips
  -> encoded_fps naturally stays at 0 during idle (also helps #20)
- handle_pw_frame entry: early return on paused, skipping ALL frame
  processing (DMA-BUF import, VAAPI scale, NV12 clone, channel send)
- MP4 mode unchanged (webrtc_paused is None, gate is no-op)
- Side benefit: last_fillable_frame stays None during initial idle,
  so maybe_send_filler_frame also early-returns -> no filler waste
  during the pre-connect idle window (partial mitigation for #18)

Out of scope (TODO comment added at state_portal.rs:178):
- Encoder still initializes on first PipeWire frame (one-time ~50ms
  SwEncEncode::new_webrtc cost). Full deferral (Option A) requires
  splitting WebRTC signaling lifecycle from media lifecycle - deferred
  until startup cost becomes user-perceptible
- Bitrate formula unchanged (5*W*H*fps/100) - tracked by #21

Verification (34s idle + 60s connected session):
- 0 'skipping duplicate frame' events during idle (was ~30/sec before)
- 0 BWE bitrate updates during idle
- 0 IDR production during idle (was 180 frames into the void)
- First IDR produced 178ms after connect (ForceKeyframe -> IDR in 10ms)
- cargo test transform/fps_limit/backend_detect: 34 passed
- SAFETY comments preserved verbatim
2026-06-20 19:57:15 +08:00
dailz 92760dd8ee fix(state_portal): make shutdown() idempotent to prevent duplicate log lines (#22)
shutdown() fired twice on exit (explicit call in main.rs + Drop impl), causing "Total: N frames" and "StatePortal shutdown complete" to log twice 23us apart. Add `shutdown_started: bool` guard at function entry.

Plain bool (not AtomicBool) because &mut self already grants exclusive access. Guard is set BEFORE cleanup so Drop re-entry during panic unwinding is suppressed.

Includes scripts/test_shutdown_idempotency.sh as a live regression test (requires Wayland session). Verified PASS on KWin: both lines print exactly once.
2026-06-20 18:45:53 +08:00
dailz 0aba0e651e fix(stats): real encode thread timing via SwEncodeTiming + take_timing (#17)
encode_thread_loop hardcoded sws_us=0 and output_bytes=0, making the
stats dashboard show zeros for those columns. The previous encode_us
was measured around the entire encode_cpu_frame call, mixing sws +
encode work into one bucket.

Add SwEncodeTiming { sws_us, encode_us, output_bytes } to SwEncEncode.
- drain_encoder returns Result<usize> (total encoded bytes), accumulated
  before the Muxer/Channel match so branch duplication and multi-packet
  drain are both handled correctly. output_bytes = bytes produced by
  libavcodec even if downstream delivery drops them.
- encode_cpu_frame resets last_timing to Default at entry (so early
  returns from disconnect/pause/dedup never report stale prior values),
  measures sws_us around sws_scale, measures encode_us around
  avcodec_send_frame + drain, then stores the complete snapshot.
- take_timing() uses mem::take to return and clear in one step.
- flush() ignores the byte count from drain_encoder.
- state_portal encode_thread_loop calls take_timing() for real values.
2026-06-14 08:50:47 +08:00
dailz 36cee9d9dd fix(webrtc): force IDR on KeyframeRequest via forced-idr=1 + pict_type (closes #16)
WebRTC KeyframeRequest 响应延迟 2.1s(152 P 帧丢弃)→ 端到端 5-20ms。

- BitrateCommand::ForceKeyframe variant over existing bitrate_tx channel
- SwEncEncode.force_keyframe_pending: bypass dedup, set AV_PICTURE_TYPE_I,
  clear only after avcodec_send_frame succeeds
- libx264 forced-idr=1 via av_opt_set (FFmpeg-level option, not x264-native)
- pict_type reset to NONE per frame (reused AVFrame leak guard)
- WebRtcInner.force_keyframe_to_encode + take_force_keyframe() drain API
- recreate_encoder sets force_keyframe_pending (resolution change → IDR)

Verification: skipping non-IDR frame 152→0, ForceKeyframe→IDR 4-5ms stable.
2026-06-14 08:20:06 +08:00
dailz 0e91c793c7 docs: add AGENTS.md project guide
Capture sources of truth, build/test instructions, runtime architecture,
and unsafe/FFI notes for AI agents working in this repo.
2026-06-13 22:46:41 +08:00
dailz 3e60258627 feat(portal): BWE-driven resolution adaptation + duplicate frame skipping
WebRTC client bandwidth estimate now drives both encoder bitrate and
resolution tier selection, replacing the previous static-target encoder.

- webrtc.rs: enable str0m BWE (seeded at 5 Mbps), surface
  EgressBitrateEstimate + KeyframeRequest events, expose
  get_bwe_estimate() / set_need_keyframe()
- state_portal.rs: wire bitrate/resolution channels between the WebRTC
  thread and the encode thread; tier ladder [1440p, 1080p, 720p] with
  downscale at 60% budget and upscale hysteresis (120% sustained 10s)
- avhw.rs: SwEncImport::poll_resolution_commands() rebuilds the import
  filter graph on UpdateResolution; SwEncEncode::recreate_encoder()
  rebuilds sws/enc_video/yuv_frame atomically; hash_sampled_y_plane()
  skips duplicate frames; VBV x264opts cap IDR bursts; H.264 level 4.0
  (muxer) / 4.2 (WebRTC)
- state.rs: sync wlr-screencopy GOP to fps*2 max 20 for parity
- fix: drain bitrate_rx + resolution_rx BEFORE the stride check in
  encode_cpu_frame() so the new (smaller-stride) frame produced after
  a resolution change does not hit the stale (larger) enc_width and
  crash the encode thread
- WebRTC GOP widened to fps*2 max 20 (was fps/2 max 10)
2026-06-13 22:46:33 +08:00
dailz 503e4dbc22 feat(portal): independent WebRTC thread + channel tuning for 60fps mouse latency
- Move WebRTC send to dedicated wl-webrtc-webrtc thread (was inline in main loop)
- Reduce frame_rx 16→1, input_tx 2→1 (drop-on-full), webrtc_tx 32→2
- Recv_timeout 10ms→2ms to reduce pipeline latency
- Fix sent_gap_p95 stats bug: compute gap at actual send time in WebRTC
  thread instead of batch-draining at snapshot time (was always 0.0ms)
- High profile via AVCodecContext.profile, veryfast preset, 5x bitrate
- Stats drain via sent_gap channel with record_send_from_thread()
- Shutdown: drop input_tx → join encode → drop webrtc_tx → join webrtc
2026-06-07 18:30:09 +08:00
dailz caccfec44e fix(portal): compositor stall detection + filler frames + PipeWire state logging
P0: Detect compositor frame delivery stalls (>100ms no frames) and log
    stall/resume events with duration. Rate-limited to 1 warn/sec.

P1: Insert duplicate raw CpuNv12Frame filler during stalls at target fps.
    Keeps WebRTC stream smooth (sent_fps 20-40 instead of 3-5 during
    compositor pauses). Stops after 2s max stale. WebRTC mode only.

P2: Replace silent _ => {} in PipeWire state_changed callback with
    explicit Paused/Streaming/Connecting log messages.

P4: Add PwCtrlEvent::FormatChanged for mid-stream dimension changes.
    param_changed detects resolution renegotiation (skips first call).
    Logs warning in poll_and_encode; full encoder reinit deferred.

Verified: cargo check 0 errors, 70/70 tests, release build, --stats live.
2026-06-07 17:20:54 +08:00
dailz 826f544569 feat(portal): async encode pipeline - decouple capture from encoding
Split synchronous encode pipeline so sws_scale + libx264 runs on a
dedicated thread, leaving only VAAPI import + GPU scale + GPU→CPU
transfer on the main capture thread.

Problem: encode_p95 occasionally hit 74ms, blocking the entire capture
pipeline and causing capture_gap_max=356ms stutter.

Solution:
- avhw.rs: Split SwEncState into SwEncImport (main thread: VAAPI import,
  filter_graph scale, GPU→CPU transfer) and SwEncEncode (encode thread:
  sws_scale NV12→YUV420P, libx264 encode). New CpuNv12Frame struct
  carries owned pixel data across threads via crossbeam channel.
  SwEncState wraps both for backward compat (MP4/sync path untouched).
- state_portal.rs: WebRTC portal path spawns 'wl-webrtc-encode' thread
  with bounded(2) input channel (drop-newest backpressure) and separate
  timing channel. Graceful shutdown: drop webrtc_rx → drop input_tx →
  join encode thread → flush sync encoder.
- stats.rs: Add record_import() + record_encode_thread() for async timing.

Results: encode_p95 stable at 2.9-4.2ms (was 11-74ms), capture_fps
stable 59-60fps, cap_gap_p95 17-19ms. Remaining capture stalls traced
to PipeWire compositor frame delivery (external, not our code).
2026-06-07 16:55:28 +08:00
dailz aae030f309 fix(webrtc): SO_SNDBUF 2MB + VBV rate limiting + stats integration
P0 - UDP send buffer: set SO_SNDBUF=2MB to prevent EAGAIN on large IDR
frames (218KB/256KB keyframes caused 18+ EAGAIN bursts). Actual Linux
buffer 4096KB confirmed.

P1 - VBV rate limiting: cap rc_max_rate=bitrate and rc_buffer_size=
bitrate/4 for WebRTC encode path, preventing oversized IDR frames.

Stats: integrate PipelineStats into cap_portal (dropped_count), state.rs
(wlroots path), webrtc.rs (browser getStats enhancement + stats panel).
2026-06-07 16:55:07 +08:00
dailz 029fe13e37 feat(stats): add --stats flag and PipelineStats windowed diagnostics
Add lightweight per-second pipeline statistics for stutter diagnosis:
- --stats CLI flag enables structured stats logging
- PipelineStats tracks capture/encode/send timing with p95/pmax
- FrameTimings records import/scale/transfer/sws/encode per-frame
- StatsSnapshot produces one structured log line per second
2026-06-07 16:54:45 +08:00
dailz f3da1e4e6c fix(webrtc): propagate poll_output error as cleanup signal to prevent zombie state (closes #14) 2026-06-06 21:48:38 +08:00
dailz e6e05fb44a fix(webrtc): fix is_idr_nalu boundary bug missing tail NAL units (closes #13) 2026-06-06 21:34:22 +08:00
dailz 8b04893ceb fix(security): remove error details from HTTP 500 response (#12)
The 500 error response previously included the raw error message {e}
in the body, potentially leaking internal implementation details (SDP
parse errors, ICE candidate info) to clients.

The detailed error is already logged server-side via tracing::error!,
so the response body is now a fixed generic string with a proper
HTTP/1.1 status line.
2026-06-06 21:22:57 +08:00
dailz 1beaea8088 fix(webrtc): use MediaAdded event to discover video mid instead of hardcoded iteration (closes #11) 2026-06-06 21:16:55 +08:00
dailz fc4733ffe8 fix: return Ok(true) on ICE Disconnected to prevent resource leak
poll_rtc() always returned Ok(false), preventing WebRtcState from
clearing self.inner on disconnect. This leaked the UDP socket, Rtc
instance, and 65KB buffer permanently if the client never reconnected.

Closes #10
2026-06-06 20:57:25 +08:00
dailz d5679be3a4 fix(state_portal): replace expect() with bail-style error propagation (closes #9) 2026-06-06 20:19:51 +08:00
dailz 36f07c92e9 fix(state_portal): prevent shutdown deadlock on full bounded channel (closes #8)
shutdown() calls enc.flush() → drain_encoder() → tx.send() on a
crossbeam bounded(32) channel.  If the channel is full and the
receiver (webrtc_rx) is alive but not being drained, send() blocks
forever — a self-deadlock since both ends belong to the same struct.

Two-layer fix:
- avhw.rs: replace tx.send() with tx.try_send(); handle Full (drop
  frame) and Disconnected (set flag) separately.
- state_portal.rs: drop webrtc_rx before flushing in shutdown() so
  try_send returns Disconnected immediately.

Regression tests added for the channel semantics.
2026-06-06 20:02:09 +08:00
dailz 7c1c9b2e19 fix(avhw): add SAFETY comments to all undocumented unsafe blocks
Close #7

- Add // SAFETY: comments to 19 undocumented unsafe blocks and impls
- Add nb_streams/null guard on stream array dereference (drain_encoder)
- Add clippy undocumented_unsafe_blocks = warn lint to prevent regression

avhw.rs now has 0 clippy unsafe documentation warnings.
2026-06-06 15:54:09 +08:00
dailz 226768c3e3 fix(avhw): handle tx.send() failure and pause encoding on WebRTC disconnect (closes #6)
- Replace 'let _ = tx.send()' with proper error handling: log warning,
  set webrtc_disconnected flag, and break drain loop on SendError
- Add Arc<AtomicBool> webrtc_paused shared between State/StatePortal
  and SwEncState, synced from wrtc.is_connected() in poll_webrtc()
- Skip encoding in encode_filtered_frame() when paused or disconnected
- Drain and discard stale channel frames on disconnect
- Resume encoding automatically on WebRTC reconnection
2026-06-06 15:12:49 +08:00
dailz fd170b66d9 fix(unsafe): add SAFETY comment and runtime guards for from_raw_parts in drain_encoder
Issue: #5

- Read AVPacket fields into local variable to avoid repeated pointer deref
- Guard against size <= 0 (prevents c_int negative wrap to huge usize)
- Guard against null data pointer (from_raw_parts(null, 0) is UB in Rust)
- Add SAFETY comment matching existing codebase convention (30+ instances)
2026-06-06 11:56:47 +08:00
dailz 9a5b09cd7f fix(security): harden token file permissions (closes #2)
- save_restore_token: use create_new(true) + mode(0o600) for exclusive
  atomic file creation, preventing symlink attacks and predictable
  temp file exploitation
- token_path: return Option, eliminate insecure /tmp fallback
- load_restore_token: reject insecure files (symlinks, wrong owner,
  group/world-readable permissions)
- Directory creation uses DirBuilderExt::mode(0o700) bypassing umask
- Added verify_secure_dir and ensure_secure_parent with full metadata
  validation (owner, permissions, symlink rejection)
- Added 11 regression tests covering all security scenarios
2026-06-06 11:05:00 +08:00
dailz 46367ef6b5 fix(state): add WebRTC support to wlr-screencopy backend
Fixes #1 -- --port mode with wlr-screencopy backend caused panic at
negotiate_format() because self.args.output is None and .expect() was
called unconditionally.

Changes:
- Introduce StreamingEncoder enum wrapping EncState (MP4) and
  SwEncState (WebRTC) with unified frames_rgb/encode_frame/flush API
- Add WebRTC fields to State<S> (webrtc, webrtc_tx, webrtc_rx,
  webrtc_frames_sent) matching Portal backend pattern
- State::new() returns Result<Self> for clean WebRtcState init failure
- negotiate_format() branches on webrtc_tx: WebRTC path uses
  SwEncState::new_webrtc(), MP4 path unchanged (hardware VAAPI)
- Add poll_webrtc() method to drive signaling + channel drain
- Event loop calls poll_webrtc() each iteration
- Fix pre-existing test/bench Args construction (Option<String> output,
  missing no_persist field)
2026-06-04 22:10:46 +08:00
dailz b0ed6548a6 feat: add WebRTC streaming via str0m + portal session persistence
- Add src/webrtc.rs: HTTP signaling server + str0m Sans-IO WebRTC transport
  with H.264 Annex-B → RTP packetization and key-frame request handling
- avhw: introduce FrameOutput enum (Muxer | Channel) so SwEncState can
  output to either MP4 muxer or crossbeam channel for WebRTC
- cap_portal: support portal session restore tokens (PersistMode::ExplicitlyRevoked)
  to skip re-authorization dialog; add --no-persist flag to force fresh dialog
- args: make --output optional when --port is used for WebRTC mode
- state_portal: integrate WebRTC pipeline (encoder channel → RTP forwarding)
  with shorter GOP for WebRTC (fps/2, min 10)
- main: redirect tracing to stderr; validate --output or --port required
- Add dependencies: str0m 0.20, serde_json 1, dirs 6
2026-06-04 20:54:16 +08:00
dailz 74f4dc826d perf(portal): achieve 58-60fps PipeWire screen capture
- Force PipeWire quantum=512 via NODE_FORCE_QUANTUM (48000/512=93Hz scheduling)
- Switch to libx264 ultrafast/zerolatency with 6 threads
- Use two-phase poll_and_encode: blocking recv_timeout for first frame,
  non-blocking try_recv drain for subsequent frames
- Remove fps_limit from portal path (PW already rate-limits via quantum/KWin;
  fps_limit's min_interval was silently dropping ~10% of valid frames)
- Remove diagnostic instrumentation (TIMING/PIPEWIRE logs, timing fields,
  pw_stats counters)
- Add lightweight production stats: per-10s fps log + shutdown summary
- Prefer libx264 over libopenh264 (better quality at same speed)
2026-05-30 08:44:15 +08:00
dailz a83d146ed3 fix: FPS limiter never passes frames when input > target rate
The old FpsLimit compared timestamps between CONSECUTIVE frames.
When PipeWire delivers at 60fps (16ms intervals) and target is 30fps
(33ms min_interval), the gap between consecutive frames is always
16ms < 33ms, so EVERY frame was rejected after the first.

Fix: track last_output_time and compare against that instead of the
previous frame's timestamp. Now frames pass when enough time has
elapsed since the last OUTPUT, not since the last INPUT.

Also adds PipeWire process callback counter logging and frame
diagnostic STATS in state_portal.rs for debugging.
2026-05-29 22:09:35 +08:00
dailz d80b34f44f feat: GPU-downscale + software H.264 encode pipeline (WIP)
Add SwEncState in avhw.rs: GPU pipeline using scale_vaapi to downscale
4K BGRA -> 2K NV12 on AMD iGPU, then software encode with libopenh264.

- import_dma_buf_to_vaapi: av_hwframe_map based DMA-BUF import
- SwEncState: GPU filter graph (scale_vaapi) + NV12->YUV420P + libopenh264
- state_portal.rs: integrated SwEncState, auto DRM device detection
- vaapi_import_bench.rs: CPU vs GPU pipeline benchmark
- sw_encode_bench.rs: software encode benchmark

Benchmark results: GPU pipeline ~91 FPS theoretical (10.95ms/frame)
vs CPU pipeline ~33 FPS (30.21ms/frame).

Known issue: only 1 frame encoded in production recording,
diagnostic STATS logging added to debug frame flow.
2026-05-29 22:04:12 +08:00
dailz 55abb5e56d fix(backend_detect): use raw zbus for portal check to avoid OnceLock connection poisoning
ashpd caches zbus::Connection in a global OnceLock. When check_portal_available()
created a Screencast proxy, the connection was cached there. When the function
returned and its tokio Runtime dropped, the cached connection became dead.
Subsequent setup_portal() calls reused this dead connection and hung forever.

Fix: replace ashpd Screencast proxy with direct zbus D-Bus interface check,
which does not touch the ashpd global connection cache.

Add examples/test_portal.rs for minimal Portal ScreenCast testing.
2026-05-27 22:07:11 +08:00
dailz 715a9c0bab refactor(cap_portal): split PwEvent into separate ctrl/frame channels
- Rename PwEvent to PwCtrlEvent, separate frame data into its own channel
- Add null chunk check to prevent crash on malformed PipeWire buffer
- Remove redundant inline comments and signal handlers
- Use try_send for error events to avoid blocking on full channel
2026-05-27 09:25:00 +08:00
dailz 60a55c17f2 fix(state_portal): add Drop impl, null dangling pointers, extract compute_pts, add tests
- Add Drop impl for StatePortal to flush encoder on drop (bug #2)
- Use enc.take() in shutdown() to prevent double-flush of write_trailer
- Null out data[0] after Box::from_raw recovery to avoid dangling pointer
- Extract compute_pts() for testable PTS calculation
- Add 8 tests: PTS calculation, DRM device resolution, descriptor building
2026-05-27 09:22:59 +08:00
dailz 5100d78aa8 fix: resolve SHM hang, DRM device mismatch, and duplicate VAAPI context
BUG-2 (HIGH): SHM Buffer event caused permanent hang
  In the ZwlrScreencopyFrameV1 dispatcher, receiving a SHM Buffer event
  left in_flight_surface stuck at AllocQueued forever, preventing
  queue_alloc_frame() from requesting new frames.
  Fix: treat Buffer as a metadata offer (v3 protocol), wait for
  BufferDone to decide failure, and add AllocQueued state guard to
  LinuxDmabuf handler.

BUG-3 (MEDIUM): Portal backend picked wrong GPU on multi-GPU systems
  state_portal.rs hardcoded /dev/dri/renderD128 then renderD129, which
  selects the wrong GPU when PipeWire uses a different device.
  Fix: extract find_drm_render_nodes() as shared utility; defer DRM
  device selection to first PipeWire frame; test each candidate with
  av_hwframe_transfer_data to find the GPU that can actually import
  the DMA-BUF frame.

BUG-4 (LOW): VAAPI device context created twice unnecessarily
  try_finalize_output() created an AvHwDevCtx stored in EverythingButFmt,
  but negotiate_format() discarded it (_hw_device_ctx) and EncState::new
  created a new one.
  Fix: thread the existing hw_device_ctx through negotiate_format() and
  create_encoder() to EncState::new() which reuses it when provided.
2026-05-25 14:32:58 +08:00
dailz 460a3ee711 fix(cap_portal): remove unsafe pw::deinit() to prevent global state corruption
pw::init() is guarded by an internal OnceCell (process-global one-shot).
pw::deinit() is unsafe and requires 'only called once per process lifetime
after all PipeWire use has permanently stopped'. Since CapPortal can be
created/destroyed multiple times, calling deinit() from a function-local
scope would prevent re-initialization (OnceCell already consumed) and
violate the unsafe contract.

The 5 early-return error paths in pipewire_thread() that previously
leaked global state are now consistent with the success path — neither
calls pw::deinit(). Process exit reclaims global PipeWire state.
2026-05-25 14:32:19 +08:00
dailz b8026981d2 feat(examples): add Wayland globals lister utility
Minimal example that connects to the Wayland compositor and prints
all advertised globals (interface name, ID, version).
2026-05-25 08:56:55 +08:00
dailz dcf8d1affb docs: add Chinese documentation comments to core modules
Add comprehensive Chinese documentation comments to cap_portal,
main, and state_portal modules covering architecture, lifecycle,
and data flow for each component.
2026-05-25 08:56:43 +08:00
dailz 25110e8463 feat(backend_detect): add portal/screencopy availability pre-checks
Add check_portal_available() and check_screencopy_available() to
probe each backend independently before committing. This enables
smarter fallback logic and better diagnostics when no backend is
found. Includes Chinese documentation comments.
2026-05-25 08:56:33 +08:00
dailz 14d1cf173a chore: add .sisyphus/ to .gitignore, remove committed BUGS.md
The .gitignore had a comment about Sisyphus artifacts but was missing
the actual pattern. BUGS.md is replaced by the .sisyphus/ ignore rule.
2026-05-25 08:56:21 +08:00
dailz 573569ade7 docs(BUGS): mark #5 zero-format encoder init as fixed (b9e62d6) 2026-05-22 15:17:39 +08:00
dailz b9e62d6830 fix(portal): reject frames before format negotiation to prevent zero-dimension encoder init 2026-05-22 15:17:14 +08:00
dailz 13ca010ecc docs(BUGS): mark #4 AVDRMFrameDescriptor leak as fixed (2d448dc) 2026-05-22 13:20:17 +08:00
dailz 2d448dcac5 fix(portal): recover AVDRMFrameDescriptor before encode_frame to prevent leak on error 2026-05-22 13:20:04 +08:00
dailz ffb36b7e0d fix(portal): convert PipeWire nanosecond PTS to encoder frame-number units
PipeWire spa_meta_header.pts is CLOCK_MONOTONIC in nanoseconds, but the
encoder expects frame-number units (time_base = 1/fps). The raw nanosecond
value was assigned directly to AVFrame.pts, causing the encoder/muxer to
interpret timestamps as billions of frames, producing corrupted duration
metadata and broken rate control.

Fix: record the first frame's PTS as a nanosecond base, compute elapsed
nanoseconds for each subsequent frame, then convert to frame numbers via
elapsed_ns * fps / 1_000_000_000. Using elapsed time avoids i64 overflow
on absolute timestamps (~10^18 ns).

Matches the WLR path pattern (state.rs:525-527) which converts microseconds
to frame numbers for the same encoder.
2026-05-22 13:05:53 +08:00
dailz 75a57e43ec docs: mark bug #2 as fixed in BUGS.md (a09a423) 2026-05-22 11:42:56 +08:00
dailz a09a4235d3 fix(portal): replace blocking send with try_send in PW process callback
PipeWire .process callback called frame_tx.send() on a bounded(3)
channel. If the encoder stalled, this blocked the PipeWire data loop,
delaying buffer recycling and potentially causing XRUNs.

Replace with try_send + AtomicU64 drop counter. Frames are silently
dropped when the channel is full (preferred for screen capture: latest
frame wins). A warning is logged every 30 dropped frames.

Fixes #2 from BUGS.md.
2026-05-22 11:42:42 +08:00
dailz 1f286b2a5d docs(BUGS): mark #1 PipeWire mainloop UAF as fixed (e40ef9e) 2026-05-22 11:31:07 +08:00
dailz e40ef9eba2 fix(cap_portal): eliminate PipeWire mainloop UAF by replacing detached helper thread with eventfd + loop-integrated add_io
The detached helper thread that called pw_main_loop_quit() through a raw
pointer cast to usize could outlive the mainloop if run() returned on its
own (event loop error, panic in callback, etc.), causing use-after-free.

Replace with an eventfd registered on the PipeWire loop via add_io(). The
shutdown callback runs on the loop thread during mainloop.run(), where the
mainloop is guaranteed alive. Drop order (reverse declaration) ensures the
IO source is unregistered before mainloop is destroyed.

Fixes: #1 (Critical UAF)
2026-05-22 11:30:12 +08:00
dailz d7fbb5256c feat: add KWin/KDE Plasma screen capture via xdg-desktop-portal ScreenCast + PipeWire
Add a second capture backend for compositors without wlr-screencopy
(KWin, GNOME, etc.) using the xdg-desktop-portal ScreenCast interface
and PipeWire DMA-BUF streaming.

New files:
- src/backend_detect.rs: auto-detect wlr-screencopy vs portal backend
- src/cap_portal.rs: Portal session setup + PipeWire DMA-BUF thread
- src/state_portal.rs: StatePortal encoder pipeline (DMA-BUF → VAAPI)

Changes:
- Cargo.toml: add ashpd 0.13, tokio 1, pipewire 0.9, libspa 0.9,
  crossbeam-channel 0.5
- src/args.rs: add --backend CLI flag
- src/avhw.rs: extract create_encoder() from inline State code
- src/main.rs: route to portal or wlr-screencopy based on backend
- src/state.rs: fix params.destroy() on dup failure, cleanup
  in_flight_surface on copy fail, use create_encoder()
- tests/integration_test.rs: add --backend flag tests
2026-05-11 08:49:08 +08:00