Commit Graph

39 Commits

Author SHA1 Message Date
dailz
46367ef6b5 fix(state): add WebRTC support to wlr-screencopy backend
Fixes #1 -- --port mode with wlr-screencopy backend caused panic at
negotiate_format() because self.args.output is None and .expect() was
called unconditionally.

Changes:
- Introduce StreamingEncoder enum wrapping EncState (MP4) and
  SwEncState (WebRTC) with unified frames_rgb/encode_frame/flush API
- Add WebRTC fields to State<S> (webrtc, webrtc_tx, webrtc_rx,
  webrtc_frames_sent) matching Portal backend pattern
- State::new() returns Result<Self> for clean WebRtcState init failure
- negotiate_format() branches on webrtc_tx: WebRTC path uses
  SwEncState::new_webrtc(), MP4 path unchanged (hardware VAAPI)
- Add poll_webrtc() method to drive signaling + channel drain
- Event loop calls poll_webrtc() each iteration
- Fix pre-existing test/bench Args construction (Option<String> output,
  missing no_persist field)
2026-06-04 22:10:46 +08:00
dailz
b0ed6548a6 feat: add WebRTC streaming via str0m + portal session persistence
- Add src/webrtc.rs: HTTP signaling server + str0m Sans-IO WebRTC transport
  with H.264 Annex-B → RTP packetization and key-frame request handling
- avhw: introduce FrameOutput enum (Muxer | Channel) so SwEncState can
  output to either MP4 muxer or crossbeam channel for WebRTC
- cap_portal: support portal session restore tokens (PersistMode::ExplicitlyRevoked)
  to skip re-authorization dialog; add --no-persist flag to force fresh dialog
- args: make --output optional when --port is used for WebRTC mode
- state_portal: integrate WebRTC pipeline (encoder channel → RTP forwarding)
  with shorter GOP for WebRTC (fps/2, min 10)
- main: redirect tracing to stderr; validate --output or --port required
- Add dependencies: str0m 0.20, serde_json 1, dirs 6
2026-06-04 20:54:16 +08:00
dailz
74f4dc826d perf(portal): achieve 58-60fps PipeWire screen capture
- Force PipeWire quantum=512 via NODE_FORCE_QUANTUM (48000/512=93Hz scheduling)
- Switch to libx264 ultrafast/zerolatency with 6 threads
- Use two-phase poll_and_encode: blocking recv_timeout for first frame,
  non-blocking try_recv drain for subsequent frames
- Remove fps_limit from portal path (PW already rate-limits via quantum/KWin;
  fps_limit's min_interval was silently dropping ~10% of valid frames)
- Remove diagnostic instrumentation (TIMING/PIPEWIRE logs, timing fields,
  pw_stats counters)
- Add lightweight production stats: per-10s fps log + shutdown summary
- Prefer libx264 over libopenh264 (better quality at same speed)
2026-05-30 08:44:15 +08:00
dailz
a83d146ed3 fix: FPS limiter never passes frames when input > target rate
The old FpsLimit compared timestamps between CONSECUTIVE frames.
When PipeWire delivers at 60fps (16ms intervals) and target is 30fps
(33ms min_interval), the gap between consecutive frames is always
16ms < 33ms, so EVERY frame was rejected after the first.

Fix: track last_output_time and compare against that instead of the
previous frame's timestamp. Now frames pass when enough time has
elapsed since the last OUTPUT, not since the last INPUT.

Also adds PipeWire process callback counter logging and frame
diagnostic STATS in state_portal.rs for debugging.
2026-05-29 22:09:35 +08:00
dailz
d80b34f44f feat: GPU-downscale + software H.264 encode pipeline (WIP)
Add SwEncState in avhw.rs: GPU pipeline using scale_vaapi to downscale
4K BGRA -> 2K NV12 on AMD iGPU, then software encode with libopenh264.

- import_dma_buf_to_vaapi: av_hwframe_map based DMA-BUF import
- SwEncState: GPU filter graph (scale_vaapi) + NV12->YUV420P + libopenh264
- state_portal.rs: integrated SwEncState, auto DRM device detection
- vaapi_import_bench.rs: CPU vs GPU pipeline benchmark
- sw_encode_bench.rs: software encode benchmark

Benchmark results: GPU pipeline ~91 FPS theoretical (10.95ms/frame)
vs CPU pipeline ~33 FPS (30.21ms/frame).

Known issue: only 1 frame encoded in production recording,
diagnostic STATS logging added to debug frame flow.
2026-05-29 22:04:12 +08:00
dailz
55abb5e56d fix(backend_detect): use raw zbus for portal check to avoid OnceLock connection poisoning
ashpd caches zbus::Connection in a global OnceLock. When check_portal_available()
created a Screencast proxy, the connection was cached there. When the function
returned and its tokio Runtime dropped, the cached connection became dead.
Subsequent setup_portal() calls reused this dead connection and hung forever.

Fix: replace ashpd Screencast proxy with direct zbus D-Bus interface check,
which does not touch the ashpd global connection cache.

Add examples/test_portal.rs for minimal Portal ScreenCast testing.
2026-05-27 22:07:11 +08:00
dailz
715a9c0bab refactor(cap_portal): split PwEvent into separate ctrl/frame channels
- Rename PwEvent to PwCtrlEvent, separate frame data into its own channel
- Add null chunk check to prevent crash on malformed PipeWire buffer
- Remove redundant inline comments and signal handlers
- Use try_send for error events to avoid blocking on full channel
2026-05-27 09:25:00 +08:00
dailz
60a55c17f2 fix(state_portal): add Drop impl, null dangling pointers, extract compute_pts, add tests
- Add Drop impl for StatePortal to flush encoder on drop (bug #2)
- Use enc.take() in shutdown() to prevent double-flush of write_trailer
- Null out data[0] after Box::from_raw recovery to avoid dangling pointer
- Extract compute_pts() for testable PTS calculation
- Add 8 tests: PTS calculation, DRM device resolution, descriptor building
2026-05-27 09:22:59 +08:00
dailz
5100d78aa8 fix: resolve SHM hang, DRM device mismatch, and duplicate VAAPI context
BUG-2 (HIGH): SHM Buffer event caused permanent hang
  In the ZwlrScreencopyFrameV1 dispatcher, receiving a SHM Buffer event
  left in_flight_surface stuck at AllocQueued forever, preventing
  queue_alloc_frame() from requesting new frames.
  Fix: treat Buffer as a metadata offer (v3 protocol), wait for
  BufferDone to decide failure, and add AllocQueued state guard to
  LinuxDmabuf handler.

BUG-3 (MEDIUM): Portal backend picked wrong GPU on multi-GPU systems
  state_portal.rs hardcoded /dev/dri/renderD128 then renderD129, which
  selects the wrong GPU when PipeWire uses a different device.
  Fix: extract find_drm_render_nodes() as shared utility; defer DRM
  device selection to first PipeWire frame; test each candidate with
  av_hwframe_transfer_data to find the GPU that can actually import
  the DMA-BUF frame.

BUG-4 (LOW): VAAPI device context created twice unnecessarily
  try_finalize_output() created an AvHwDevCtx stored in EverythingButFmt,
  but negotiate_format() discarded it (_hw_device_ctx) and EncState::new
  created a new one.
  Fix: thread the existing hw_device_ctx through negotiate_format() and
  create_encoder() to EncState::new() which reuses it when provided.
2026-05-25 14:32:58 +08:00
dailz
460a3ee711 fix(cap_portal): remove unsafe pw::deinit() to prevent global state corruption
pw::init() is guarded by an internal OnceCell (process-global one-shot).
pw::deinit() is unsafe and requires 'only called once per process lifetime
after all PipeWire use has permanently stopped'. Since CapPortal can be
created/destroyed multiple times, calling deinit() from a function-local
scope would prevent re-initialization (OnceCell already consumed) and
violate the unsafe contract.

The 5 early-return error paths in pipewire_thread() that previously
leaked global state are now consistent with the success path — neither
calls pw::deinit(). Process exit reclaims global PipeWire state.
2026-05-25 14:32:19 +08:00
dailz
b8026981d2 feat(examples): add Wayland globals lister utility
Minimal example that connects to the Wayland compositor and prints
all advertised globals (interface name, ID, version).
2026-05-25 08:56:55 +08:00
dailz
dcf8d1affb docs: add Chinese documentation comments to core modules
Add comprehensive Chinese documentation comments to cap_portal,
main, and state_portal modules covering architecture, lifecycle,
and data flow for each component.
2026-05-25 08:56:43 +08:00
dailz
25110e8463 feat(backend_detect): add portal/screencopy availability pre-checks
Add check_portal_available() and check_screencopy_available() to
probe each backend independently before committing. This enables
smarter fallback logic and better diagnostics when no backend is
found. Includes Chinese documentation comments.
2026-05-25 08:56:33 +08:00
dailz
14d1cf173a chore: add .sisyphus/ to .gitignore, remove committed BUGS.md
The .gitignore had a comment about Sisyphus artifacts but was missing
the actual pattern. BUGS.md is replaced by the .sisyphus/ ignore rule.
2026-05-25 08:56:21 +08:00
dailz
573569ade7 docs(BUGS): mark #5 zero-format encoder init as fixed (b9e62d6) 2026-05-22 15:17:39 +08:00
dailz
b9e62d6830 fix(portal): reject frames before format negotiation to prevent zero-dimension encoder init 2026-05-22 15:17:14 +08:00
dailz
13ca010ecc docs(BUGS): mark #4 AVDRMFrameDescriptor leak as fixed (2d448dc) 2026-05-22 13:20:17 +08:00
dailz
2d448dcac5 fix(portal): recover AVDRMFrameDescriptor before encode_frame to prevent leak on error 2026-05-22 13:20:04 +08:00
dailz
ffb36b7e0d fix(portal): convert PipeWire nanosecond PTS to encoder frame-number units
PipeWire spa_meta_header.pts is CLOCK_MONOTONIC in nanoseconds, but the
encoder expects frame-number units (time_base = 1/fps). The raw nanosecond
value was assigned directly to AVFrame.pts, causing the encoder/muxer to
interpret timestamps as billions of frames, producing corrupted duration
metadata and broken rate control.

Fix: record the first frame's PTS as a nanosecond base, compute elapsed
nanoseconds for each subsequent frame, then convert to frame numbers via
elapsed_ns * fps / 1_000_000_000. Using elapsed time avoids i64 overflow
on absolute timestamps (~10^18 ns).

Matches the WLR path pattern (state.rs:525-527) which converts microseconds
to frame numbers for the same encoder.
2026-05-22 13:05:53 +08:00
dailz
75a57e43ec docs: mark bug #2 as fixed in BUGS.md (a09a423) 2026-05-22 11:42:56 +08:00
dailz
a09a4235d3 fix(portal): replace blocking send with try_send in PW process callback
PipeWire .process callback called frame_tx.send() on a bounded(3)
channel. If the encoder stalled, this blocked the PipeWire data loop,
delaying buffer recycling and potentially causing XRUNs.

Replace with try_send + AtomicU64 drop counter. Frames are silently
dropped when the channel is full (preferred for screen capture: latest
frame wins). A warning is logged every 30 dropped frames.

Fixes #2 from BUGS.md.
2026-05-22 11:42:42 +08:00
dailz
1f286b2a5d docs(BUGS): mark #1 PipeWire mainloop UAF as fixed (e40ef9e) 2026-05-22 11:31:07 +08:00
dailz
e40ef9eba2 fix(cap_portal): eliminate PipeWire mainloop UAF by replacing detached helper thread with eventfd + loop-integrated add_io
The detached helper thread that called pw_main_loop_quit() through a raw
pointer cast to usize could outlive the mainloop if run() returned on its
own (event loop error, panic in callback, etc.), causing use-after-free.

Replace with an eventfd registered on the PipeWire loop via add_io(). The
shutdown callback runs on the loop thread during mainloop.run(), where the
mainloop is guaranteed alive. Drop order (reverse declaration) ensures the
IO source is unregistered before mainloop is destroyed.

Fixes: #1 (Critical UAF)
2026-05-22 11:30:12 +08:00
dailz
d7fbb5256c feat: add KWin/KDE Plasma screen capture via xdg-desktop-portal ScreenCast + PipeWire
Add a second capture backend for compositors without wlr-screencopy
(KWin, GNOME, etc.) using the xdg-desktop-portal ScreenCast interface
and PipeWire DMA-BUF streaming.

New files:
- src/backend_detect.rs: auto-detect wlr-screencopy vs portal backend
- src/cap_portal.rs: Portal session setup + PipeWire DMA-BUF thread
- src/state_portal.rs: StatePortal encoder pipeline (DMA-BUF → VAAPI)

Changes:
- Cargo.toml: add ashpd 0.13, tokio 1, pipewire 0.9, libspa 0.9,
  crossbeam-channel 0.5
- src/args.rs: add --backend CLI flag
- src/avhw.rs: extract create_encoder() from inline State code
- src/main.rs: route to portal or wlr-screencopy based on backend
- src/state.rs: fix params.destroy() on dup failure, cleanup
  in_flight_surface on copy fail, use create_encoder()
- tests/integration_test.rs: add --backend flag tests
2026-05-11 08:49:08 +08:00
dailz
2972216a02 fix(avhw): specify format=nv12 output for scale_vaapi filter
scale_vaapi defaults to the input sw_format (RGBZ) when no output format
is specified. h264_vaapi encoder only supports NV12/YUV formats.
Adding format=nv12 ensures the filter outputs the correct color format
for hardware encoding.
2026-04-15 15:01:06 +08:00
dailz
1065364261 fix(avhw): add missing mut on video_filter for buffersink access 2026-04-15 14:41:58 +08:00
dailz
833cbdfed3 chore(avhw): clean up debug tracing in VA-API pipeline 2026-04-15 14:01:22 +08:00
dailz
c77838235a fix(avhw): derive encoder hw_frames_ctx from filter graph buffersink 2026-04-15 13:57:34 +08:00
dailz
10ee190fd2 fix(state): do not destroy screencopy proxy on SHM buffer event 2026-04-15 07:15:46 +08:00
dailz
68514bd3f6 fix(state): do not set errored on SHM buffer event
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-15 07:12:33 +08:00
dailz
506e5ea30e fix(state,avhw): bind initial globals manually + fix filter graph crash on niri
- registry_queue_init consumes registry events during its internal
  roundtrip without forwarding them to Dispatch<WlRegistry>. Added
  bind_initial_globals() to manually iterate GlobalList and bind all
  initial globals (wl_output, xdg_output_manager, dmabuf, screencopy,
  wlr_output_manager) at State::new time.
- Fix av_freep segfault in build_filter_graph: av_buffersrc_parameters_alloc
  returns a plain pointer, use av_free instead of av_freep (which expects
  pointer-to-pointer).
- Fix filter graph format negotiation: remove software format filter that
  broke scale_vaapi hardware pipeline. Chain is now src -> scale -> sink.
- Downgrade repeat_pps error to warning (not available in FFmpeg 6.x).
2026-04-14 20:42:05 +08:00
dailz
b2e5f37cf6 feat(state): support compositors without xdg-output via wlr-output-management
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-14 17:52:59 +08:00
dailz
ecd78492ee feat(avhw): integrate transform into VA-API filter graph
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-14 17:02:54 +08:00
dailz
e89689634d fix(state): handle BufferParamsEvent::Failed to prevent silent hang
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-14 16:50:48 +08:00
dailz
6835f1f6cd fix(main): call fps_limit.flush() before encoder flush on shutdown
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-04-14 16:45:47 +08:00
dailz
3a954dae1a fix: resolve 4 audit bugs (PTS, WlOutput, first_frame, DRM minor) 2026-04-06 16:25:02 +08:00
dailz
27aa8d2c65 fix: convert PTS from from frame号单位转换 2026-04-06 16:23:59 +08:00
dailz
1e8d00126b fix(bsf): remove non-functional BSF pipeline, use encoder repeat_pps option 2026-04-06 09:35:37 +08:00
dailz
6d49222de8 feat: Phase 1 MVP with audit fixes — Wayland screen capture + VAAPI encoding
Phase 1 MVP implementation of wl-webrtc: Wayland screen capture tool
with hardware-accelerated VAAPI H.264 encoding and WebTransport output.

Includes all 9 runtime bug fixes from code audit (fix-audit-issues plan):

CRITICAL:
- C2: h264_metadata BSF with repeat_sps/repeat_pps in encode pipeline
- C4: FpsLimit wired as timing gate in on_copy_complete

HIGH:
- C3+A2: DRM device discovery via dmabuf feedback MainDevice event,
  unified resolve_drm_path() helper (CLI > compositor > auto > fallback)
- H2: Separate physical_size (mm) from mode_size (pixels) in wl_output
- H1+A3: Multi-output warning + named-output-not-found error

MEDIUM:
- M5: tv_sec u32->u64 to avoid Y2106 timestamp truncation
- M4: Guard against SHM Buffer event (DMA-BUF only)

Key components:
- src/avhw.rs: FFmpeg VAAPI encoder + filter graph + BSF pipeline
- src/state.rs: Wayland event loop + output negotiation + screencopy
- src/cap_wlr_screencopy.rs: wlr-screencopy capture source
- src/fps_limit.rs: Frame rate limiting with configurable target
- src/transform.rs: Frame format conversion utilities
2026-04-05 23:35:00 +08:00