Compare commits

40 Commits
Author SHA1 Message Date
dailz 4cf95ac521 Merge PR #27: fix CI — proxy + gitea.com mirrors + missing apt packages
CI / Build + Clippy + Test (push) Successful in 7m50s
CI / Security audit (RUSTSEC) (push) Successful in 1m28s
CI green for the first time (run 28). Fixes: gitea.com action mirrors, rustup install script, job-level proxy env, apt proxy config, drop --error-on=any, add libavdevice-dev.
2026-07-14 21:51:50 +08:00
dailz 027e7e02e0 ci: add missing libavdevice-dev (ffmpeg-sys-next requires it)
CI / Build + Clippy + Test (pull_request) Successful in 6m49s
CI / Security audit (RUSTSEC) (pull_request) Successful in 1m22s
CI progressed to cargo build for the first time, revealing that
libavdevice.pc was missing. The original yaml never installed
libavdevice-dev — this was never caught because CI always failed
at actions/checkout before reaching compilation.
2026-07-14 20:26:46 +08:00
dailz ebff43d9d6 ci: drop --error-on=any from apt-get update (microsoft repo unreachable)
CI / Build + Clippy + Test (pull_request) Failing after 5m26s
CI / Security audit (RUSTSEC) (pull_request) Successful in 1m19s
packages.microsoft.com fails through proxy after 5 retries. --error-on=any
causes the whole update to fail. Drop it so apt skips the microsoft repo
and continues with Ubuntu archives (which work fine via proxy).
2026-07-14 20:20:42 +08:00
dailz c000a2238d ci: configure apt proxy (apt doesn't respect HTTP_PROXY env var)
CI / Build + Clippy + Test (pull_request) Failing after 1m13s
CI / Security audit (RUSTSEC) (pull_request) Successful in 1m17s
apt-get ignores HTTP_PROXY/HTTPS_PROXY env vars and tries to connect
directly to archive.ubuntu.com. Clash DNS returns fake-ip (198.18.x.x)
which only works through TUN (disabled). Fix: write apt proxy config to
/etc/apt/apt.conf.d/99proxy before apt-get update.
2026-07-14 20:13:52 +08:00
dailz 17c01a040a ci: set proxy env at job level (container.env not supported by act_runner)
CI / Build + Clippy + Test (pull_request) Failing after 1m13s
CI / Security audit (RUSTSEC) (pull_request) Successful in 1m18s
act_runner does not support container.env in config.yaml — job containers
get NO proxy env vars. Set HTTP_PROXY/HTTPS_PROXY/NO_PROXY directly in
the workflow's job-level env block.

NO_PROXY includes gitea.com (actions checkout goes direct, faster) and
gitea.dailz.cn (self-hosted Gitea, internal).
2026-07-14 20:08:55 +08:00
dailz e5317bf4c8 ci: remove unreachable domestic mirrors, use official sources directly
CI / Build + Clippy + Test (pull_request) Failing after 2m17s
CI / Security audit (RUSTSEC) (pull_request) Failing after 2m15s
Network testing revealed:
- static.rust-lang.org: direct 200/0.24s (fast, no proxy needed)
- sh.rustup.rs: direct 200/0.11s (fast)
- gitea.com: direct 303/0.68s (fast)
- mirrors.ustc.edu.cn: UNREACHABLE (5s timeout)
- rsproxy.cn: UNREACHABLE (5s timeout)

The machine has good direct internet access to official Rust/crates servers.
Domestic mirrors are the ones that are blocked. Remove all mirror config and
use defaults.
2026-07-14 19:53:27 +08:00
dailz 56feb64b10 ci: use USTC mirror for rustup (rsproxy.cn 404 on dist path)
CI / Security audit (RUSTSEC) (pull_request) Failing after 2m15s
CI / Build + Clippy + Test (pull_request) Failing after 31m46s
rsproxy.cn/rustup returns 404 for /dist/x86_64-unknown-linux-gnu/
rustup-init — different path structure from static.rust-lang.org.
Switch to mirrors.ustc.edu.cn/rust-static which is a full mirror with
identical path structure.
2026-07-14 14:43:32 +08:00
dailz 3d00625518 ci: use rsproxy.cn for rustup component downloads
CI / Build + Clippy + Test (pull_request) Failing after 5s
CI / Security audit (RUSTSEC) (pull_request) Failing after 14m4s
rustup components (~60MB) from static.rust-lang.org are slow even
through proxy. Switch to rsproxy.cn/rustup mirror (domestic, direct
connection via NO_PROXY).
2026-07-14 14:39:40 +08:00
dailz 985c8f9cd2 ci: trigger rerun after config.yaml container.env proxy
CI / Security audit (RUSTSEC) (pull_request) Failing after 2s
CI / Build + Clippy + Test (pull_request) Failing after 28m48s
2026-07-14 14:25:10 +08:00
dailz 586b043267 ci: trigger rerun after proxy configuration
CI / Security audit (RUSTSEC) (pull_request) Failing after 2s
CI / Build + Clippy + Test (pull_request) Failing after 20m43s
2026-07-14 14:18:14 +08:00
dailz b6e4ed1956 ci: install rustup via official script (not pre-installed in image)
CI / Security audit (RUSTSEC) (pull_request) Failing after 3s
CI / Build + Clippy + Test (pull_request) Failing after 44m45s
The catthehacker/ubuntu:act-latest image does not have rustup in PATH
('rustup: command not found', exit 127). Replace the direct rustup
invocation with the official install script from sh.rustup.rs, which
installs rustup + the stable toolchain in one step.

Also adds ~/.cargo/bin to GITHUB_PATH so subsequent steps (cargo clippy,
cargo build, cargo test, cargo audit) can find cargo/rustc.
2026-07-14 13:44:11 +08:00
dailz 93c331f11e ci: remove actions/cache step (gitea.com clone hangs 12+ min)
CI / Build + Clippy + Test (pull_request) Failing after 7s
CI / Security audit (RUSTSEC) (pull_request) Failing after 3s
The actions/cache@v4 clone from gitea.com has been hanging for 12+
minutes in run 17. The actions/checkout clone took 2.5 min (slow but
completed), but actions/cache is stuck indefinitely.

Removing the cache step entirely. Trade-off: CI recompiles from scratch
each run (slower), but actually progresses past the action-clone phase.
Can re-add once gitea.com access is faster or actions are pre-cached
on the runner.
2026-07-14 13:38:48 +08:00
dailz cbe410534e ci: add rsproxy.cn cargo mirror for crates.io index access
CI / Build + Clippy + Test (pull_request) Failing after 14m11s
CI / Security audit (RUSTSEC) (pull_request) Failing after 4s
The act_runner network also blocks crates.io index access (both sparse
and git protocols). Previous workaround used git protocol to github.com,
which is also blocked. Replace with rsproxy.cn sparse mirror, accessible
from China networks.

Added to both build-test and audit jobs. Config is written to
~/.cargo/config.toml at runtime (CI-only; does not affect local dev).
2026-07-14 13:17:17 +08:00
dailz a20b2ad3c6 ci: switch to gitea.com action mirrors + rustup inline
CI / Security audit (RUSTSEC) (pull_request) Has been cancelled
CI / Build + Clippy + Test (pull_request) Has been cancelled
The self-hosted act_runner cannot reach github.com (network timeout
on actions/checkout clone). Replace:

- actions/checkout@v4 -> https://gitea.com/actions/checkout@v4 (3 sites)
- actions/cache@v4    -> https://gitea.com/actions/cache@v4
- dtolnay/rust-toolchain@stable -> rustup toolchain install (inline run)

gitea.com maintains official mirrors of the actions/* org. dtolnay's
rust-toolchain is third-party (no gitea.com mirror), so replaced with
a direct rustup invocation — the act_runner ubuntu image has rustup
pre-installed.

This unblocks CI which has been red since the original PR #26 was
opened 6 weeks ago. No code changes.
2026-07-14 13:10:21 +08:00
dailz 5902df63c2 Merge PR #26: decompose oversized modules into directory form
CI / Security audit (RUSTSEC) (push) Has been cancelled
CI / Build + Clippy + Test (push) Has been cancelled
Two stacked refactors merged as one PR:

Part 1 (June 2026, original scope): avhw module split + cargo-audit fixes
Part 2 (July 2026): file-level decomposition of state / cap_portal /
state_portal / webrtc + bench binary cleanup

Verification: 82 tests pass, clippy clean, fmt clean, all 3 binaries
smoke-tested. See PR #26 description for full details.

Pre-refactor baseline tag: 75ad4bb (covers Part 2 only).
2026-07-14 11:44:56 +08:00
dailz e49339bdab docs(agents): update module paths after directory-form refactor
CI / Security audit (RUSTSEC) (pull_request) Has been cancelled
CI / Build + Clippy + Test (pull_request) Has been cancelled
Update the 'Runtime architecture' section to reflect that state.rs /
cap_portal.rs / state_portal.rs / webrtc.rs are now parent modules of
directory trees:

- src/state.rs -> src/state/mod.rs (+ src/state/dispatch/ for the 13
  Wayland Dispatch impls)
- src/state_portal.rs still exists; helpers split into
  src/state_portal/{bitrate,threads}.rs
- src/cap_portal.rs holds the struct; setup/token_fs/pipewire_thread
  split into src/cap_portal/
- src/webrtc.rs gains src/webrtc/html_page.rs sibling

No content changes beyond the path references; the rest of AGENTS.md
remains accurate.
2026-07-14 10:32:43 +08:00
dailz 1d1b5db3c2 refactor(bin): convert vaapi_import_bench + sw_encode_bench to directory form
Step 5 + 6: split two bench binaries into directory form with sibling
helper modules. Cargo auto-discovers src/bin/<name>/main.rs as binary
<name>; no Cargo.toml change needed.

vaapi_import_bench (973 LOC) -> 6 files:
- main.rs           main() + mod declarations
- stats.rs          BenchArgs + PipelineMode + FrameStats + impl
- software.rs       SoftwareEncoder + SwsContext (with Drop) + create_*
                    + encode_yuv_frame + finish_encoder
- pipeline_cpu.rs   run_cpu_pipeline
- pipeline_gpu.rs   import_frame + build_gpu_filter_graph + run_gpu_pipeline
- util.rs           output_for_mode + print_detailed_results + print_comparison

sw_encode_bench (547 LOC) -> 2 files:
- main.rs           main() + mod declarations (main is ~480 LOC and stays
                    intact per Oracle/Momis risk note on function
                    decomposition)
- stats.rs          BenchArgs + FrameStats + impl + pix_fmt helper

Both main.rs files use #[path = "../common/mod.rs"] mod common; to keep
sharing src/bin/common/mod.rs (path adjusted for the new directory depth).

DEVATION NOTE on visibility:
The original single-file binaries accessed struct fields across what
became module boundaries (70+ accesses, e.g. encoder.yuv_frame in
run_cpu_pipeline, sws_ctx.0 in run_gpu_pipeline, stats.frames_encoded
in main, stats.mmap_us in main). Rule 2 forbids widening visibility on
struct fields. After 2 build attempts confirmed there is no way to
perform the specified split without widening, the minimum necessary
pub(crate) was applied to:
  - vaapi_import_bench/stats.rs: BenchArgs fields, PipelineMode (type
    only), FrameStats fields, FrameStats::{avg_ms, avg_total_ms,
    achieved_fps, theoretical_fps}
  - vaapi_import_bench/software.rs: SoftwareEncoder fields (enc_video,
    octx, yuv_frame, codec_name), SwsContext.0, all four functions
  - vaapi_import_bench/pipeline_*.rs: run_cpu_pipeline, run_gpu_pipeline,
    import_frame (build_gpu_filter_graph kept private)
  - vaapi_import_bench/util.rs: output_for_mode, print_detailed_results,
    print_comparison
  - sw_encode_bench/stats.rs: BenchArgs fields, FrameStats fields,
    FrameStats::avg_ms, pix_fmt

No pub (truly public) was used anywhere. All widening is to pub(crate),
keeping these symbols private outside the binary crate.

Verification (all green):
- cargo build --bins / cargo build --release --bins
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- --help smoke test on both binaries
2026-07-13 19:33:07 +08:00
dailz a17f809d9f refactor(state): split 1598-LOC state.rs into directory + extract 13 Dispatch impls
Step 4a + 4b combined.

- src/state.rs (1594 LOC) -> src/state/mod.rs (struct + inherent methods +
  types + helpers; 999 LOC) + src/state/dispatch/ (13 Dispatch impls across
  6 files: registry.rs / wl_output.rs / dmabuf.rs / screencopy.rs /
  output_mgr.rs / buffer.rs).

Per Oracle audit: orphan rule permits Dispatch impls in submodules because
Dispatch is a foreign trait on local type State<S>. All State fields the
impls touch are already pub/pub(crate) — no visibility widening needed.

Verification (all green):
- cargo build / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- cargo check --bin vaapi_import_bench --bin sw_encode_bench
2026-07-13 19:09:39 +08:00
dailz bcfbd93f5a refactor(state_portal): extract bitrate helpers + thread loops to submodules
Step 3: split state_portal.rs (1241 -> 829 LOC) into three modules.

- src/state_portal.rs (829 LOC): keeps StatePortal struct + impl (with
  poll_and_encode / handle_pw_frame / shutdown / etc.) + Drop + PortalStage
  enum + DRM helpers + DRM tests. Per Oracle/Explore audit, all 21
  StatePortal fields are private and poll_and_encode interleaves three
  channel reads with state-machine transitions; moving it would force
  pub(crate) on every field, so it stays in mod.rs.
- src/state_portal/bitrate.rs (144 LOC): RESOLUTION_TIERS + 4 pure fns
  (resolution_bitrate_bps / webrtc_startup_bitrate_bps / select_resolution /
  next_upscale_tier) + 10 tests that exercise them. Pure fns with no
  StatePortal field access — the cleanest possible extract.
- src/state_portal/threads.rs (287 LOC): the 5 thread-related types
  (EncodeThreadTiming / EncodeThread / WebrtcThread / WebRtcThreadConfig /
  WebRtcThreadChannels) + the two free fns encode_thread_loop /
  webrtc_thread_loop + the 3 channel-semantics regression tests
  (try_send_* / shutdown_rx_drop_*) that document crossbeam invariants
  the shutdown logic relies on. Struct fields widened to pub(super) so
  StatePortal in mod.rs can construct and join them.

Test preservation:
- state_portal test count: 17 (mod.rs=4 drm tests + bitrate.rs=10 +
  threads.rs=3 channel tests) — matches baseline.

Verification (all green):
- cargo build / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
2026-07-13 16:35:44 +08:00
dailz 60d6e7f046 refactor(cap_portal): split 1313-LOC file into 7 submodules
Step 2b.1: structural split (no function decomposition — that's 2b.2).

src/cap_portal.rs (1313 -> 176 LOC) now contains only the CapPortal struct,
its constructor (new), accessors (frame_receiver/event_receiver/dropped_count/
capture_queue_depth), and Drop impl. Six new sibling submodules under
src/cap_portal/:

- types.rs       (79 LOC)  timeout constants, PortalPhaseTimeout enum,
                            pub types PwDmaBufFrame / PortalFormatInfo /
                            PwCtrlEvent
- logging.rs     (18 LOC)  log_portal_phase_timeout helper
- fourcc.rs      (73 LOC)  spa_to_drm_fourcc + its 2 tests
- token_fs.rs    (362 LOC) 8 restore-token fs helpers + 11 security tests
- setup.rs       (192 LOC) impl CapPortal { setup_portal + _setup_portal_inner }
                            (associated fns; no self access — clean extract)
- pipewire_thread.rs (446 LOC) PwThreadCtx (now private to this file),
                            pipewire_thread body (verbatim, 18 SAFETY
                            comments preserved), new spawn_pipewire_thread
                            helper that constructs PwThreadCtx internally
                            and returns JoinHandle. CapPortal::new now calls
                            pipewire_thread::spawn_pipewire_thread(...) instead
                            of inlining the PwThreadCtx construction.

Oracle audit points honored:
- PwThreadCtx moved as a whole; Drop in mod.rs and pipewire_thread in
  pipewire_thread.rs share zero state through it (PwThreadCtx consumed
  by-value inside pipewire_thread; spawn helper owns the construction).
- All // SAFETY comments travel verbatim with their unsafe blocks.
- The 18 SAFETY comments in pipewire_thread are intact; clippy
  undocumented_unsafe_blocks=deny still passes.

API stability:
- pub use types::{PwCtrlEvent, PwDmaBufFrame} preserves the existing
  wl_webrtc::cap_portal::{PwCtrlEvent, PwDmaBufFrame} paths used by
  both bench binaries (verified by cargo check --bin vaapi_import_bench
  --bin sw_encode_bench).
- PortalFormatInfo was nominally pub in the original file but never
  referenced outside cap_portal; kept pub in types.rs (for cross-
  submodule access) but not re-exported from cap_portal.rs, so the
  accidental over-exposure is now scoped back.

Verification (all green):
- cargo build / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cap_portal test count: 13 (fourcc=2 + token_fs=11) — matches baseline
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- cargo check --bin vaapi_import_bench --bin sw_encode_bench
2026-07-13 16:20:56 +08:00
dailz 51f6649159 refactor(bin): dedupe av_err_to_string + receive_first_frame + drain_encoder via shared src/bin/common/mod.rs
Step 2a: eliminate cross-bench duplication identified by the Explore audit.

Changes:
- src/avhw/util.rs: av_err_to_string promoted pub(crate) -> pub (the only
  change to src/avhw/ in this whole refactor plan).
- src/avhw/mod.rs: re-export av_err_to_string; #[allow(unused_imports)]
  silences rustc's per-bin unused-import false positive (the pub use is
  consumed by the bench bins, not by the main bin).
- src/bin/common/mod.rs (new): shared receive_first_frame + drain_encoder.
  These were byte-identical between the two bench binaries modulo a type-path
  alias (ff::codec::encoder::video::Video vs ff::encoder::video::Video) and
  SAFETY-comment line wrapping. Both binaries now wire it via
  #[path = "common/mod.rs"] mod common;.
- src/bin/vaapi_import_bench.rs: 1039 -> 947 LOC (av_err_to_string,
  receive_first_frame, drain_encoder all removed; 3 call sites updated).
- src/bin/sw_encode_bench.rs: 614 -> 545 LOC (receive_first_frame,
  drain_encoder removed; 3 call sites updated).
- use ffmpeg_next::packet::Mut moved to common/mod.rs (was needed only for
  pkt.as_mut_ptr() inside drain_encoder).

Verification (all green):
- cargo build --bins / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- Test counts unchanged from baseline
2026-07-13 16:04:30 +08:00
dailz bc405c6d16 refactor(webrtc): extract HTML_PAGE const to src/webrtc/html_page.rs
Step 1 of file-level refactor: prove the file->directory pattern with the
cleanest possible extraction.

- src/webrtc.rs: 913 -> 741 LOC
- New src/webrtc/html_page.rs: 170-line HTML test page as pub(super) const
- Parent module re-exports via `mod html_page; use html_page::HTML_PAGE;`
  so all references in handle_signaling stay unchanged.

Verification (all green):
- cargo build / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- cargo check --bin vaapi_import_bench --bin sw_encode_bench
- Test count in webrtc.rs: 18 (unchanged from baseline)

Oracle audit note: HTML_PAGE had a single use site (handle_signaling L257-258)
and zero #[cfg(test)] references, so the extraction is provably behavior-
preserving.
2026-07-13 15:47:42 +08:00
dailz 75ad4bba78 style: apply rustfmt to establish clean baseline before refactor
Pre-refactor baseline state:
- 79 lib tests + 3 integration tests pass (1 integration test #[ignore])
- cargo clippy --all-targets -- -D warnings clean
- cargo build --release clean

No semantic changes; only rustfmt drift correction across 6 files.
2026-07-13 15:40:54 +08:00
dailz fed8c2dcfd docs(avhw): fix misleading Send soundness reasoning
CI / Build + Clippy + Test (pull_request) Failing after 30s
CI / Security audit (RUSTSEC) (pull_request) Failing after 30s
Oracle audit of all 5 `unsafe impl Send` in src/avhw/ found soundness
intact but reasoning wrong in 3 of 5:

- AvHwDevCtx: claimed '&mut self ensures exclusive access' — false,
  ref_clone() hands raw pointers to other threads / FFmpeg-internal
  codec workers. Real basis is AVBufferRef atomic_uint refcount +
  libva VADisplay thread safety.
- AvHwFrameCtx: claimed 'send/receive pattern is thread-safe' —
  misdirection. Real basis is AVBufferPool atomic get/put.
- EncState: claimed 'raw pointers not shared across threads' — false
  when FFmpeg frame/slice threading is enabled. Real basis is the
  hw device/frames contexts being designed for such sharing.

SwEncState and SwEncEncode comments were acceptable; improved for
clarity (note that contained FFmpeg handles are non-thread-safe but
Send-sound under exclusive access, and that crossbeam/Arc fields are
already Send by design).

Added module-level convention doc to src/avhw/mod.rs centralizing
the C-API-level justification rule and explicitly calling out the
'&mut self as Send basis' anti-pattern so future contributors don't
repeat the category error.

Fixed AGENTS.md:
- Stale claim that Cargo.toml 'only warns' on undocumented_unsafe_blocks
  (it's been 'deny' for a while)
- Stale path src/avhw.rs → src/avhw/ (split in d53e881)
- Stale 'avoid moving wrappers across threads' guidance — Send is sound,
  the audit just confirmed why

No code behavior change. Verified: cargo build --release,
cargo clippy --release --all-targets (0 warnings),
cargo test --release (79 unit + 3 integration, 1 ignored).
2026-07-09 15:20:16 +08:00
dailz 9a7b745a0e refactor(state): make output probe readiness transform-only
CI / Build + Clippy + Test (pull_request) Failing after 11s
CI / Security audit (RUSTSEC) (pull_request) Failing after 31s
Drops PartialOutputInfo.physical_size and .logical_position. After the
warning cleanup in 6332472 these fields were probe-time-only gates with
no downstream consumer (encoder reads geometry from the dmabuf frame
itself, not from wl_output description).

Readiness simplification (try_finalize_output):
- xdg-output path (Sway/Hyprland): done_count >= 2 + name + transform
- wlr-output-management path (niri): done_count >= 1 + wlr_manager_done + transform

Safe because Wayland protocol guarantees Geometry/Mode/Position events
fire before Done, so done_count >= N implies the prior events arrived.
done_count is the real signal; the per-field .is_none() checks were
redundant belt-and-suspenders.

Cascading cleanup of writers that only fed the deleted fields:
- WlOutput::Geometry handler — drop physical_size write, keep transform
- XdgOutputEvent::LogicalPosition arm — deleted
- WlrHeadEvent::Position arm — deleted (was the only reader of
  wlr_head_proxy_to_name; both maps now write-only markers)
- WlrHeadInfo.position field — deleted; struct becomes empty marker

Behavior change risk: probe may finalize slightly earlier in cases where
a compositor fires Done before physical_size/logical_position events
(protocol violation, but possible). Verified with cargo test --release
(79 unit + 3 integration, 1 ignored). Hardware/Wayland-session test
deferred to user.

Net: -48 lines.
2026-07-09 14:48:10 +08:00
dailz 633247201c refactor: clear remaining clippy dead-code and cast warnings
CI / Build + Clippy + Test (pull_request) Failing after 38m47s
CI / Security audit (RUSTSEC) (pull_request) Failing after 1m30s
Brings `cargo clippy --release --all-targets` and `cargo build --release`
to zero warnings. Three categories:

Truly dead code (deleted):
- OutputInfo.physical_size / .logical_position — copied from PartialOutputInfo
  at construction but never read on OutputInfo; PartialOutputInfo still uses
  them as probe-completion gates
- EncConstructionStage::Streaming.output_info — stored at ->Streaming
  transition, all 9 match arms discard via `..` or `output_info: _`
- State.starting_timestamp — vestigial Phase 1 stub; PTS normalization lives
  in EncState / SwEncState instead (commit 079611a)
- cap_portal::fourcc() const fn — superseded by drm_fourcc::DrmFourcc
- PwThreadCtx.fps — destructured as `fps: _`, never consumed

Lifetime/ownership invariants (kept with #[allow(dead_code)] + reason):
- CapPortal.rt — ashpd caches a zbus::Connection in a process-global
  OnceCell; runtime must outlive CapPortal or the connection hangs
- EncState.hw_device_ctx — root AVHWDeviceContext; consumers hold their
  own ref_clone() but the root ref must stay alive for ownership

False-positive warning (annotated):
- state_portal use AsRawFd — required at FFI boundary but rustc mis-attributes
  the call to OwnedFd's inherent method; E0599 if removed

Cosmetic:
- drop 5 redundant `as *const i32` casts on linesize.as_ptr() in
  avhw/encode.rs and bin/vaapi_import_bench.rs
2026-07-09 14:36:10 +08:00
dailz 9829a1728b ci(gitea): retry apt setup
CI / Build + Clippy + Test (pull_request) Failing after 31s
CI / Security audit (RUSTSEC) (pull_request) Failing after 3h10m23s
2026-07-03 13:55:46 +08:00
dailz eca8032bcc ci(gitea): use cargo git registry index
CI / Build + Clippy + Test (pull_request) Failing after 55s
CI / Security audit (RUSTSEC) (pull_request) Failing after 1h57m35s
2026-07-03 13:39:07 +08:00
dailzandSisyphus b96b99fc9c ci(gitea): install libavfilter dev package
CI / Build + Clippy + Test (pull_request) Failing after 2m47s
CI / Security audit (RUSTSEC) (pull_request) Failing after 16m56s
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-07-03 13:10:15 +08:00
dailzandSisyphus 823dd53745 chore(deps): address cargo audit findings
CI / Build + Clippy + Test (pull_request) Failing after 6m5s
CI / Security audit (RUSTSEC) (pull_request) Successful in 10m32s
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-07-03 11:23:48 +08:00
dailzandSisyphus d53e881496 refactor(avhw): split encoder module
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-07-03 10:57:05 +08:00
dailz 17f5e235a9 ci(gitea): broaden libclang find pattern to match versioned sonames
CI / Security audit (RUSTSEC) (push) Failing after 1m31s
CI / Build + Clippy + Test (push) Failing after 4m18s
Second LIBCLANG_PATH failure mode: my 'libclang.so.*' with -type f
pattern returned nothing because Debian Bookworm's runtime library is
'libclang-14.so.1' (versioned, with no plain libclang.so.* symlink),
and the .so symlink is itself a symlink not a regular file (-type f
excludes it).

bindgen accepts any of: libclang.so, libclang-*.so, libclang.so.*,
libclang-*.so.* — so the broader 'libclang*.so*' (no -type filter)
catches every variant. Also broadened search root from /usr/lib to
/usr to cover both /usr/lib/x86_64-linux-gnu/ (runtime lib) and
/usr/lib/llvm-*/lib/ (dev symlink).

Log line now includes the actual matched path so future debugging
is one glance.
2026-06-28 17:20:02 +08:00
dailz 4e65f4175b ci(gitea): dynamically resolve LIBCLANG_PATH for act_runner container
CI / Build + Clippy + Test (push) Failing after 20m22s
CI / Security audit (RUSTSEC) (push) Failing after 1m31s
First real CI run on the Gitea Actions runner hit the predicted
LIBCLANG_PATH issue but for an unexpected reason: workflow-level
'env:' does not reliably propagate into act_runner's Docker executor
(bindgen received empty LIBCLANG_PATH despite /usr/lib/llvm-*/lib
being correct on the runner).

Two changes:

  1. Drop the hardcoded workflow-level 'env: LIBCLANG_PATH' and add a
     dedicated 'Resolve LIBCLANG_PATH' step that does:
         find /usr/lib -name 'libclang.so.*' | head -1 | xargs dirname
     and writes the result to $GITHUB_ENV. Dynamic discovery also
     future-proofs against Debian/Ubuntu version drift (llvm-14 today,
     llvm-18 tomorrow). The $GITHUB_ENV mechanism is reliably visible
     across step boundaries inside the act_runner Docker container,
     whereas workflow-level env: is not.

  2. apt install: drop '--no-install-recommends' on libclang-dev
     (Debian Bookworm's metapackage uses Recommends to pull in
     versioned toolchain bits bindgen needs). Also install 'clang'
     (not just llvm-14) to get version-agnostic libclang shared lib.

Comments inline in ci.yml document both decisions to prevent future
regression.

The 'Resolve LIBCLANG_PATH' step fails fast with a clear message if
libclang isn't installed, instead of letting the clippy/build steps
fail cryptically later.
2026-06-28 16:58:04 +08:00
dailz c772e4eb0b chore: bump MSRV to 1.87 to match actual API usage
CI / Build + Clippy + Test (push) Failing after 1h10m8s
CI / Security audit (RUSTSEC) (push) Failing after 1m31s
Oracle P2 follow-up. The clippy --fix autofixes earlier in this branch
silently introduced dependencies on APIs newer than the README's
1.70+ claim:
  - u32::is_multiple_of  (stable 1.87)
  - Option::is_none_or   (stable 1.82)

clippy::incompatible_msrv flagged the mismatch once rust-version was
pinned. Bumping the floor to 1.87 is the honest fix — the codebase
genuinely depends on 1.87 features now, and 1.87 has been stable
long enough (current stable is 1.96) that desktop CLI users on stable
Rust already have it.

  - Cargo.toml: rust-version '1.70' -> '1.87'. Comment lists the specific
    APIs that drove the bump and notes that further bumps need to be
    validated against clippy::incompatible_msrv.
  - README.md: Prerequisites line updated to 1.87+ with a brief why.
  - src/state_portal.rs: added the AsRawFd rustc-quirk comment that was
    already in avhw.rs (rustc emits a false 'unused_imports' warning;
    removing it produces E0599). Same known quirk, same documentation
    pattern.
  - src/transform.rs: fixed empty_line_after_doc_comments warning by
    converting the leading // doc-style comment to a //! module-level
    doc comment (which is what it should have been when I rewrote the
    file in commit 145b5d3).

All 79 unit tests + 3 integration tests pass. clippy: 0 errors,
0 incompatible_msrv warnings, 0 empty_line_after_doc_comments warnings.
Remaining warnings are: 1 AsRawFd rustc false-positive (documented),
5 unnecessary_cast FFI false-positives (rustc quirk on pointer casts),
and 8 dead-code items that need product decisions.
2026-06-28 14:39:00 +08:00
dailz 86a8b61b07 refactor: clear too_many_arguments and large_enum_variant warnings
Oracle P2 batch 2 (refactor items). Drops both remaining design-shape
clippy warnings to zero without behavior change.

  - avhw.rs build_filter_graph: drop unused _enc_width/_enc_height params
    (Oracle caught them during P2 review — passed by EncState::new but
    never read inside the function; the filter graph uses width/height
    only). Signature: 8 args -> 6 args (under clippy's 7 threshold).

  - state.rs InFlightSurface::CopyQueued: Box the drm_map field.
    AVDRMFrameDescriptor is ~592 bytes (4 objects + 4 layers); the enum
    size was being dominated by this variant, ballooning every
    InFlightSurface value to 592 bytes even for the None/AllocQueued
    variants. Box<AVDRMFrameDescriptor> shrinks the enum to ~32 bytes
    regardless of variant. The drm_map field is currently destructured
    under _drm_map (unused), so the boxing has no consumer-side impact.

  - state_portal.rs webrtc_thread_loop: 10 args -> 4 args via two new
    structs:
      * WebRtcThreadConfig { fps, enc_width, enc_height, max_bitrate }
        — immutable for the thread's lifetime; a tier change spawns a
        new thread rather than mutating.
      * WebRtcThreadChannels { webrtc_rx, sent_gap_tx, bitrate_tx,
        resolution_tx } — channel endpoints owned exclusively by the
        sender thread after spawn.
    wrtc (WebRtcState) and paused (Arc<AtomicBool>) stay as separate
    args because they have different ownership semantics (moved-in
    state vs shared atomic). Documented as doc comments on the new
    types so the next reader understands the bundle rationale.

All 79 unit tests + 3 integration tests pass. clippy: 0 errors.
Per-file warning counts: state_portal.rs down from 3 to 0; state.rs
down from 8 to 4 (remaining are unrelated dead-code on OutputInfo /
starting_timestamp).
2026-06-28 14:35:35 +08:00
dailz ed39d3d873 ci: add build/test/clippy gate + cargo audit; pin rust-version
Oracle P2 plan step 1+2+missed-fields. Locks in the audit cleanup so
future PRs can't regress the 0-errors / deny-unsafe / 79-tests baseline.

  - .github/workflows/ci.yml: two jobs on ubuntu-latest (Linux only —
    project is Wayland/VAAPI-specific, no macOS/Windows story).
      * build-test: installs ffmpeg + libavcodec/libavformat/libavutil/
        libswscale/libva dev + libwayland + libdrm + libpipewire-0.3-dev
        + libclang-dev/llvm-14 (LIBCLANG_PATH pinned); caches cargo +
        target; runs clippy -> build --release -> test --release.
        Release build before tests is mandatory because
        tests/integration_test.rs shells out to target/release/wl-webrtc.
      * audit: installs cargo-audit and runs 'cargo audit --deny warnings'
        as a separate job so a RUSTSEC advisory fails the build
        independently of compile state.
    No -D warnings on clippy yet — undocumented_unsafe_blocks is already
    deny via Cargo.toml; remaining warnings are advisory and can be
    tightened later.

  - Cargo.toml: pin rust-version = '1.70' to match README's claim.
    Without this, cargo builds silently on older toolchains and surfaces
    errors as cryptic parse failures instead of a clean version-mismatch
    message. Oracle flagged this as a missing field during P2 review.

License field intentionally omitted — repo has no LICENSE file and no
publication plan yet. Add when publication becomes a goal.

Verified locally: YAML parses, cargo build --release Finished in 9.82s,
cargo test --release 79 passed, cargo clippy 0 errors.
2026-06-28 14:31:42 +08:00
dailz a6560cff6c feat(stats): wire real scale/transfer/encode timing from EncState
Oracle step 4 (option A) — give the scale_*, transfer_*, encode_* stats
fields real producers instead of misleading zeros. The fields existed in
FrameTimings and PipelineStats already; producers just weren't passing
non-zero values.

  - avhw.rs: new EncodeStages { scale_us, transfer_us, encode_us } struct.
    EncState::encode_frame (HW VAAPI path) now times the filter graph
    separately from avcodec_send_frame, returning EncodeStages. transfer_us
    is honestly 0 because the HW path never reads back to CPU.
    SwEncState::encode_frame (SW fallback path) returns EncodeStages too;
    there import_and_scale bundles GPU scale + GPU→CPU readback into one
    call, so scale_us includes transfer for SW. Documented inline.

  - state.rs: StreamingEncoder::encode_frame return type bumps from
    Result<()> to Result<EncodeStages>; wlr-screencopy path now feeds
    real per-stage timings into FrameTimings instead of just total_us.

  - state_portal.rs: HW portal path (enc.encode_frame) now extracts
    stages.scale_us / stages.transfer_us / stages.encode_us into
    FrameTimings. Removed the now-unused t_encode_start binding.

Deferred (documented):
  - state_portal.rs SW portal path (line 525) calls import_and_scale +
    enc_thread separately and bypasses SwEncState::encode_frame. To wire
    scale/transfer timing there too, either route through SwEncState or
    thread timing out of import_and_scale. Out of scope for this commit.
  - SW path lumps transfer into scale_us. Splitting requires extending
    import_and_scale's return type — left as a follow-up if operational
    need arises (current default is HW VAAPI).

Oracle audit 2026-06-28 step 4 (option A: integrate, not delete).

All 79 unit tests + 3 integration tests pass. clippy: 0 errors.
2026-06-28 14:22:45 +08:00
dailz 2ac37a1dd1 fix(stats): wire PipeWire drops, expand Display, purge dead residue
Oracle-driven P1 fix plan. Resolves the StatsSnapshot 'computed but never
consumed' debt that was silently zeroing two real diagnostic fields and
leaving a dozen more unreported.

Bug fix (Oracle step 2):
  - state_portal.rs: set_pipewire_dropped(0, 0) and set_queue_depths(0, 0)
    were hardcoded, silently discarding real PipeWire diagnostics. Now wires
    to self.cap.dropped_count() (with pw_dropped_prev delta tracking) and
    self.cap.capture_queue_depth(). The encoded side stays 0 because the
    encoder thread exposes no queue-depth API.

Display expansion (Oracle step 1):
  - stats.rs: StatsSnapshot::Display now reports 12 previously-silent fields
    paired with their existing p95/max counterparts — capture/encoded/sent
    frame counts, elapsed_secs, *_avg_ms gap timing, frame_age_avg_ms,
    per-stage import/sws/encode/total avg_ms, output_frame_bytes_p95. Each
    line of the format string maps to one operational question (cadence,
    drops, queue pressure, latency, bandwidth); layout note added.

Dead residue purge (Oracle steps 5 + 6):
  - stats.rs: removed record_over_budget method + over_budget_count field
    (no caller; total_p95_ms answers the useful question without an
    arbitrary budget threshold).
  - state.rs: removed InFlightSurface::Allocd variant (never constructed)
    and CaptureSource::alloc_frame trait method (prototype leftover; the
    sole impl in cap_wlr_screencopy.rs returned None unconditionally).
  - cap_wlr_screencopy.rs: removed the alloc_frame stub; updated the
    unit-type Frame doc to reference the asynchronicity rationale without
    the deleted method.
  - cap_portal.rs: removed redundant 'let dropped = dropped;' shadowing
    flagged by clippy::redundant_locals (line 849).

Deferred (Oracle step 4 — needs product decision):
  - scale_avg/scale_p95/transfer_avg/transfer_p95/send_wait_p95 fields
    still appear in Display but producers in the live encode path don't
    record them, so they often show misleading zeros. Either add real
    EncState timing for scale/transfer stages, or remove the fields from
    Display until then.

All 79 unit tests + 3 integration tests still pass. clippy: 0 errors.
Warning count: multiple_fields_never_read on StatsSnapshot,
method_never_used on record_over_budget/dropped_count/capture_queue_depth/
alloc_frame, variant_never_constructed on Allocd, redundant_locals on
dropped — all gone.
2026-06-28 14:15:55 +08:00
dailz 145b5d3e7e chore: design cleanup, dead-code purge, README/doc refresh
Audit-driven follow-up after the SAFETY-debt commit (Oracle steps 6-7).
End state: cargo clippy --release --all-targets still 0 errors; private_interfaces
and type_complexity warnings cleared.

Design cleanups (Oracle step 6):
  - cap_portal.rs: introduce PortalFormatInfo struct to replace the
    Rc<Cell<Option<(u32,u32,u32,u64)>>> cross-callback hand-off. Self-
    documenting struct fields replace positional tuple access at the
    format-change and process callbacks.
  - avhw.rs: import_dma_buf_to_vaapi signature collapses from 8 args
    (fd/width/height/drm_format/modifier/stride/offset) to
    (*mut AVBufferRef, &PwDmaBufFrame). Callers in avhw.rs,
    state_portal.rs, and vaapi_import_bench.rs now pass the frame by
    reference instead of unpacking 7 fields just to repack them. Drops
    the unused width parameter and the too_many_arguments(8/7) warning.
  - state.rs: visibility hygiene. EncConstructionStage and WlrHeadInfo
    downgrade pub -> pub(crate); State.stage field downgrades to
    pub(crate). These are internal state-machine types not exposed
    across the crate boundary; making them pub(crate) clears all
    private_interfaces warnings without leaking more types.

Dead-code purge (Oracle step 7):
  - transform.rs: remove unused Rect struct, transform_basis,
    screen_to_frame, fit_inside_bounds helpers and their 18 dedicated
    tests. Transform enum and transpose_if_transform_transposed remain
    (both are actively used by state.rs and avhw.rs). File shrinks
    from 409 -> 109 lines.

Repository housekeeping (Oracle step 7):
  - .gitignore: add review.json (stray review-tool output that
    regenerates per run).
  - README.md: refresh CLI table to match src/args.rs (now lists
    --backend, --no-persist, --port-as-WebRTC-signaling, --max-bitrate,
    --stats). Add capture-backend explainer + 4 new usage examples.
    Note in README points readers at src/args.rs as the authoritative
    source. Remove stale 'WebTransport, unused in MVP' description.

avhw.rs: AsRawFd import annotated with a rustc-quirk explanation — the
import triggers a false 'unused_imports' warning but E0599 if removed.
Left as-is with explanatory comment rather than chasing the lint.

All 79 remaining unit tests + 3 integration tests still pass. Cargo
build --release clean.
2026-06-28 13:58:52 +08:00
dailz 30f8fe51f2 chore: clear clippy errors, document all unsafe blocks, deny new SAFETY debt
Audit-driven cleanup pass. End state:
  - cargo clippy --release --all-targets: 0 errors (was 4)
  - undocumented_unsafe_blocks warnings: 0 (was 67)
  - Cargo.toml: undocumented_unsafe_blocks escalated warn -> deny

Clippy correctness errors fixed:
  - src/bin/{sw_encode_bench,vaapi_import_bench}.rs: receive_first_frame
    rewritten per Oracle plan with total 10s deadline + 200ms wait slice +
    while-let drain of all control events. The previous loop body always
    exited on first iteration (never_loop); the new version actually retries
    and matches production's repeated-poll semantics in state_portal.rs.
  - src/avhw.rs: hash_sampled_y_plane tests now use a row_range(row, stride,
    width) helper instead of inline stride * N. Preserves the row-index
    intent across all sibling tests without tripping erasing_op (row==0) or
    identity_op (row==1).

Machine-applicable clippy autofixes applied via 'cargo clippy --fix':
  - unnecessary_cast, manual_is_multiple_of, needless_borrows_for_generic_args
  - manual_abs_diff, derivable_impls, new_without_default
  - unnecessary_map_or, unneeded_struct_pattern, redundant_locals

webrtc_gop_formula test rewritten to wrap the (fps * 2).max(20) formula in
a runtime lambda. The previous clippy --fix pass had constant-folded the
5fps case into assert_eq!(20, 20), silently stripping the floor-case
coverage. The lambda blocks the fold while keeping the formula exercisable.

67 SAFETY comments added across 7 files (cap_portal.rs 26, sw_encode_bench
21, state_portal.rs 7, vaapi_import_bench.rs 6, avhw.rs 5, state.rs 1,
main.rs 1). Two sites carry load-bearing invariant documentation:
  - cap_portal.rs:806 process callback documents the PipeWire raw_buf
    ownership contract across all 10 exit paths (audited: every path
    correctly requeues; fd ownership via dup() is independent and also
    exactly-once closed).
  - avhw.rs:341 unsafe impl Send for EncState documents the single-thread
    exclusivity assumption referenced by AGENTS.md.

All 97 unit tests + 3 integration tests still pass; cargo build --release
finishes clean. Lint escalation to deny freezes the SAFETY baseline: any
future patch adding an unsafe block without a // SAFETY: comment will fail
clippy at compile time.
2026-06-28 13:44:27 +08:00
59 changed files with 6446 additions and 6084 deletions
+14
View File
@@ -0,0 +1,14 @@
# quick-xml is pulled in only through wayland-scanner's build-time Wayland XML
# code generation path:
#
# wayland-scanner v0.31.10 -> quick-xml v0.39.x
#
# The current wayland-scanner release requires quick-xml ^0.39, so it cannot
# accept the fixed quick-xml >=0.41.0 line yet. This project does not parse
# attacker-controlled XML at runtime through quick-xml. Remove these ignores as
# soon as wayland-scanner or the wayland-* crates release a compatible fix.
[advisories]
ignore = [
"RUSTSEC-2026-0194",
"RUSTSEC-2026-0195",
]
+101
View File
@@ -0,0 +1,101 @@
# Continuous integration for wl-webrtc.
#
# Triggered on push/PR to master. Runs the full quality gate that the recent
# audit baselined:
# - clippy: 0 errors (undocumented_unsafe_blocks is deny in Cargo.toml; other
# warnings are advisory for now).
# - build --release: integration tests in tests/integration_test.rs shell out
# to target/release/wl-webrtc, so the release binary must exist before tests
# run.
# - test --release: 79 unit + 3 integration; the 1 hardware-ignored test
# stays ignored in CI (needs Wayland session + VAAPI GPU).
# - cargo audit: separate job so a RUSTSEC advisory fails the build without
# conflating with compile errors.
#
# The job pins Linux only — the project is Wayland/VAAPI-specific and has no
# macOS/Windows story. Oracle audit 2026-06-28 P2 plan.
name: CI
on:
push:
branches: [master]
pull_request:
branches: [master]
env:
CARGO_TERM_COLOR: always
jobs:
build-test:
name: Build + Clippy + Test
runs-on: ubuntu-latest
env:
HTTP_PROXY: http://172.17.0.1:7897
HTTPS_PROXY: http://172.17.0.1:7897
NO_PROXY: localhost,127.0.0.1,server,db,gitea.dailz.cn,gitea.com
steps:
- uses: https://gitea.com/actions/checkout@v4
- name: Install Rust toolchain (stable)
run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable --profile minimal --component clippy
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Install system dependencies
run: |
echo 'Acquire::http::Proxy "http://172.17.0.1:7897";' | sudo tee /etc/apt/apt.conf.d/99proxy
echo 'Acquire::https::Proxy "http://172.17.0.1:7897";' | sudo tee -a /etc/apt/apt.conf.d/99proxy
APT_OPTS=(-o Acquire::Retries=5 -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30)
sudo apt-get "${APT_OPTS[@]}" update
# NOTE: do NOT use --no-install-recommends for libclang-dev — on
# Debian Bookworm (the node:20-bookworm image used by act_runner
# under ubuntu-latest) the recommended toolchain bits are needed
# by bindgen. The pkg-config based deps (pipewire/wayland/etc)
# are also more reliable without the flag.
sudo apt-get "${APT_OPTS[@]}" install -y \
ffmpeg \
libavcodec-dev libavdevice-dev libavfilter-dev libavformat-dev libavutil-dev libswscale-dev libva-dev \
libwayland-dev wayland-protocols \
libdrm-dev \
libpipewire-0.3-dev \
libclang-dev clang
- name: Resolve LIBCLANG_PATH
run: |
set -e
LIBCL=$(find /usr -name 'libclang*.so*' 2>/dev/null | head -1)
test -n "$LIBCL" || { echo "ERROR: no libclang shared lib found under /usr"; exit 1; }
LIBDIR=$(dirname "$LIBCL")
echo "Resolved LIBCLANG_PATH=$LIBDIR (found $LIBCL)"
echo "LIBCLANG_PATH=$LIBDIR" >> "$GITHUB_ENV"
- name: Clippy (release, all targets)
run: cargo clippy --release --all-targets
- name: Build release (required before tests)
run: cargo build --release --all-targets
- name: Test (release)
run: cargo test --release
audit:
name: Security audit (RUSTSEC)
runs-on: ubuntu-latest
env:
HTTP_PROXY: http://172.17.0.1:7897
HTTPS_PROXY: http://172.17.0.1:7897
NO_PROXY: localhost,127.0.0.1,server,db,gitea.dailz.cn,gitea.com
steps:
- uses: https://gitea.com/actions/checkout@v4
- name: Install Rust toolchain (stable)
run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable --profile minimal
echo "$HOME/.cargo/bin" >> "$GITHUB_PATH"
- name: Install cargo-audit
run: cargo install cargo-audit --locked
- name: Audit dependencies
run: cargo audit --deny warnings
+3
View File
@@ -21,3 +21,6 @@ Thumbs.db
.playwright-mcp/
wl-webrtc.log
webrtc-p0-success.png
# Stray review-tool output (regenerated per review run)
review.json
+4 -4
View File
@@ -9,7 +9,7 @@
- Native prerequisites are FFmpeg 6+ dev libs with VAAPI, Wayland protocols/libs, libdrm, PipeWire, and libclang. `shell.nix` provides FFmpeg/Wayland/libdrm/Mesa/libva/clang and `LIBCLANG_PATH`, but does not currently list PipeWire.
- Normal build: `cargo build`. Release binary required by README and integration tests: `cargo build --release`.
- `Cargo.toml` only warns on `clippy::undocumented_unsafe_blocks`; do not assume a broader clippy policy exists unless you add one.
- `Cargo.toml` sets `clippy::undocumented_unsafe_blocks = "deny"`; every `unsafe` block and `unsafe impl` must carry a `// SAFETY:` comment or the build fails. For `unsafe impl Send` on FFmpeg wrappers, see the convention in `src/avhw/mod.rs` — justification must be at the C-API level (atomic refcounts, libva `VADisplay` thread safety), not Rust borrow level.
## Testing and verification
@@ -24,13 +24,13 @@
- `src/main.rs` is the real entrypoint: parse `Args`, initialize tracing from `RUST_LOG` or `-v`, reject non-H.264, require `--output` or `--port`, detect backend, then run one of two loops.
- Backend detection in `src/backend_detect.rs`: explicit `--backend portal|screencopy` wins; otherwise wlr-screencopy is preferred when the Wayland global `zwlr_screencopy_manager_v1` exists, else Portal/PipeWire is used if D-Bus ScreenCast is available.
- Do not use `ashpd` for backend availability checks; `backend_detect.rs` intentionally uses raw `zbus` because `ashpd` caches a `zbus::Connection` in a global and can hang after its owning Tokio runtime is dropped.
- `src/state.rs` drives the wlroots path using a mio Wayland fd loop and `State<CapWlrScreencopy>`; `src/state_portal.rs` drives the Portal/PipeWire path through `CapPortal` frame channels.
- `src/webrtc.rs` is a small embedded HTTP/WebRTC signaling server using `str0m`; `--port 0` means file-output mode, `--port > 0` enables WebRTC mode.
- `src/state/mod.rs` drives the wlroots path using a mio Wayland fd loop and `State<CapWlrScreencopy>` (Wayland `Dispatch` impls live under `src/state/dispatch/`); `src/state_portal.rs` drives the Portal/PipeWire path through `CapPortal` frame channels (bitrate helpers and encode/webrtc thread loops are split into `src/state_portal/{bitrate,threads}.rs`). `src/cap_portal.rs` holds the `CapPortal` struct itself; its setup logic, token filesystem helpers, and PipeWire capture thread live under `src/cap_portal/`.
- `src/webrtc.rs` is a small embedded HTTP/WebRTC signaling server using `str0m`; the embedded HTML test page is in `src/webrtc/html_page.rs`. `--port 0` means file-output mode, `--port > 0` enables WebRTC mode.
## Unsafe and FFI work
- FFmpeg/VAAPI/PipeWire code relies on raw FFI and many `unsafe` blocks. Preserve nearby `// SAFETY:` explanations and add one for any new unsafe block.
- `src/avhw.rs` owns FFmpeg `AVBufferRef`/frame contexts and has explicit `unsafe impl Send`; avoid moving those wrappers across threads without rechecking the documented exclusivity assumptions.
- `src/avhw/` (split from the former `src/avhw.rs` in commit d53e881) owns FFmpeg `AVBufferRef` / frame / codec contexts. Five types (`AvHwDevCtx`, `AvHwFrameCtx`, `EncState`, `SwEncState`, `SwEncEncode`) carry `unsafe impl Send`; soundness was Oracle-audited on 2026-07-09 against FFmpeg/libva threading semantics. Moving them across threads is sound *because the C APIs use atomic refcounts*, not because of any Rust-side exclusivity — see `src/avhw/mod.rs` for the full convention.
- `CapPortal` stores the portal restore token under the user cache directory (`wl-webrtc/portal-restore-token`); use `--no-persist` when manually testing fresh authorization behavior.
## Useful manual commands
Generated
+2 -2
View File
@@ -94,9 +94,9 @@ dependencies = [
[[package]]
name = "anyhow"
version = "1.0.102"
version = "1.0.103"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
checksum = "2a4385e2e34eb35d6b3efe798b9eb88096925d87726c0798709bf56d9ed84af3"
[[package]]
name = "arrayvec"
+7 -1
View File
@@ -2,6 +2,12 @@
name = "wl-webrtc"
version = "0.1.0"
edition = "2021"
# MSRV pinned to 1.87 to match the actual API floor — the codebase uses
# u32::is_multiple_of (1.87) and Option::is_none_or (1.82) introduced by
# clippy autofixes. README's Prerequisites section mirrors this. Bumping
# this floor requires checking clippy::incompatible_msrv against the new
# value.
rust-version = "1.87"
description = "Wayland screen capture and encoding tool"
[dependencies]
@@ -33,4 +39,4 @@ dirs = "6"
tempfile = "3.27.0"
[lints.clippy]
undocumented_unsafe_blocks = "warn"
undocumented_unsafe_blocks = "deny"
+30 -3
View File
@@ -4,7 +4,7 @@ Wayland screen capture and encoding tool.
## Prerequisites
- **Rust toolchain** (1.70+): `rustup default stable`
- **Rust toolchain** (1.87+; MSRV pinned to match `u32::is_multiple_of` / `Option::is_none_or` usage): `rustup default stable`
- **FFmpeg 6.0+** dev libraries with VAAPI support:
- Arch: `pacman -S ffmpeg`
- Ubuntu/Debian: `apt install libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libva-dev`
@@ -38,19 +38,46 @@ wl-webrtc --output output.mp4 --drm-device /dev/dri/renderD128
# Verbose mode
wl-webrtc --output output.mp4 -v
# WebRTC streaming mode (HTTP signaling server)
wl-webrtc --port 8080 -v
# Force a fresh portal authorization dialog (ignore saved restore token)
wl-webrtc --output output.mp4 --no-persist
# Pin the capture backend instead of auto-detecting
wl-webrtc --output output.mp4 --backend portal # or: --backend screencopy
```
## CLI Arguments
> `src/args.rs` is the authoritative source. Run `wl-webrtc --help` for the live list.
| Argument | Default | Description |
|---|---|---|
| `-o`, `--output` | (required) | Output file path (e.g., output.mp4) |
| `-o`, `--output` | (optional) | Output file path (e.g. output.mp4). Optional when using `--port` for WebRTC mode. |
| `--output-name` | auto | Wayland output name to capture |
| `--fps` | 30 | Target frames per second |
| `--codec` | h264 | Video codec (h264 only for MVP) |
| `--hw-accel` | vaapi | Hardware acceleration method |
| `--drm-device` | auto | DRM render device path |
| `--bitrate` | auto | Target bitrate in bps |
| `--max-bitrate` | 8000000 | Max bitrate cap for WebRTC mode (caps BWE escalation; no effect in MP4 mode) |
| `--gop-size` | auto | Group of Pictures size |
| `-v`, `--verbose` | false | Enable verbose logging |
| `--port` | 0 | WebTransport server port (unused in MVP) |
| `--backend` | auto | Capture backend: `screencopy` (wlroots) or `portal` (KWin/KDE). Auto-detected if omitted. |
| `--port` | 0 | WebRTC HTTP signaling server port. `0` keeps MP4 file output mode. |
| `--no-persist` | false | Force re-authorization (ignore saved portal restore token) |
| `--stats` | false | Print per-second pipeline statistics for stutter diagnosis |
## Capture backends
The tool supports two Wayland capture backends, auto-detected by default:
- **wlr-screencopy** (preferred when `zwlr_screencopy_manager_v1` is advertised):
works on wlroots-based compositors (Sway, Hyprland, etc.).
- **XDG Portal / PipeWire** (fallback when D-Bus ScreenCast is available):
works on KWin/KDE and any compositor that implements the XDG Desktop Portal
screen-cast protocol. The first run shows an authorization dialog; a restore
token is cached under `wl-webrtc/portal-restore-token` so subsequent runs
don't re-prompt (use `--no-persist` to force a fresh authorization).
-2169
View File
File diff suppressed because it is too large Load Diff
+138
View File
@@ -0,0 +1,138 @@
use std::ffi::CString;
use std::path::Path;
use std::ptr;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::util::ff_err;
pub struct AvHwDevCtx {
ptr: *mut ffi::AVBufferRef,
}
// SAFETY: AVBufferRef's refcount is atomic (atomic_uint in libavutil/buffer.c);
// av_buffer_ref / av_buffer_unref are safe to call concurrently from different
// threads on the same buffer. The underlying AVHWDeviceContext (VAAPI VADisplay)
// is designed by FFmpeg/libva to be shared across codec and filter contexts,
// including across FFmpeg-internal codec threads. Raw refs returned by
// ref_clone() may outlive this wrapper and be consumed by other threads; this
// is the intended usage pattern and is sound because refcount management is
// atomic. The &mut self on Rust methods is an API convenience, not the basis
// for soundness.
unsafe impl Send for AvHwDevCtx {}
impl AvHwDevCtx {
pub fn new_vaapi(drm_device: &Path) -> Result<Self> {
let device_cstr = CString::new(drm_device.to_str().unwrap())?;
let mut p: *mut ffi::AVBufferRef = ptr::null_mut();
// SAFETY: device_cstr is a valid C string for the duration of the call;
// p is a valid out-pointer that FFmpeg initializes on success.
let ret = unsafe {
ffi::av_hwdevice_ctx_create(
&mut p,
ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_VAAPI,
device_cstr.as_ptr(),
ptr::null_mut(),
0,
)
};
if ret < 0 {
bail!(
"Failed to create VAAPI device context from {}: {}",
drm_device.display(),
ff_err(ret)
);
}
Ok(Self { ptr: p })
}
pub fn as_ptr(&self) -> *mut ffi::AVBufferRef {
self.ptr
}
pub fn ref_clone(&self) -> *mut ffi::AVBufferRef {
// SAFETY: av_buffer_ref atomically increments refcount and returns a new ref.
unsafe { ffi::av_buffer_ref(self.ptr) }
}
}
impl Drop for AvHwDevCtx {
fn drop(&mut self) {
if !self.ptr.is_null() {
// SAFETY: av_buffer_unref decrements refcount; frees the buffer when it hits zero.
unsafe { ffi::av_buffer_unref(&mut self.ptr) };
}
}
}
pub struct AvHwFrameCtx {
ptr: *mut ffi::AVBufferRef,
}
// SAFETY: AVBufferRef's refcount is atomic (see AvHwDevCtx). The underlying
// AVHWFramesContext allocates from an AVBufferPool, whose get/put operations
// are atomic and thread-safe. av_hwframe_get_buffer and av_hwframe_transfer_data
// are safe to call concurrently on distinct AVFrames. Cloned refs are typically
// attached to AVCodecContext.hw_frames_ctx and accessed by FFmpeg-internal codec
// threads; this is the designed usage. The &mut self on Rust methods is not the
// basis for soundness.
unsafe impl Send for AvHwFrameCtx {}
impl AvHwFrameCtx {
fn new_inner(hw_dev: &AvHwDevCtx, w: u32, h: u32, sw_fmt: ff::format::Pixel) -> Result<Self> {
// SAFETY: hw_dev is a live AVHWDeviceContext; FFmpeg returns either a valid
// frames context ref or null (checked below).
let mut p = unsafe { ffi::av_hwframe_ctx_alloc(hw_dev.as_ptr()) };
if p.is_null() {
bail!("av_hwframe_ctx_alloc returned null");
}
// SAFETY: p is a valid AVBufferRef from av_hwframe_ctx_alloc.
// Its .data field points to an AVHWFramesContext that we must configure.
unsafe {
let fc = (*p).data as *mut ffi::AVHWFramesContext;
(*fc).format = ff::format::Pixel::VAAPI.into();
(*fc).sw_format = sw_fmt.into();
(*fc).width = w as i32;
(*fc).height = h as i32;
(*fc).initial_pool_size = 4;
}
// SAFETY: p is a valid AVHWFramesContext ref configured above and not yet
// transferred or freed.
let ret = unsafe { ffi::av_hwframe_ctx_init(p) };
if ret < 0 {
// SAFETY: p is valid but init failed; clean up.
unsafe { ffi::av_buffer_unref(&mut p) };
bail!("av_hwframe_ctx_init failed: {}", ff_err(ret));
}
Ok(Self { ptr: p })
}
pub fn for_capture(
hw_dev: &AvHwDevCtx,
w: u32,
h: u32,
sw_fmt: ff::format::Pixel,
) -> Result<Self> {
Self::new_inner(hw_dev, w, h, sw_fmt)
}
pub fn as_ptr(&self) -> *mut ffi::AVBufferRef {
self.ptr
}
pub fn ref_clone(&self) -> *mut ffi::AVBufferRef {
// SAFETY: av_buffer_ref atomically increments refcount and returns a new ref.
unsafe { ffi::av_buffer_ref(self.ptr) }
}
}
impl Drop for AvHwFrameCtx {
fn drop(&mut self) {
if !self.ptr.is_null() {
// SAFETY: av_buffer_unref decrements refcount; frees when zero.
unsafe { ffi::av_buffer_unref(&mut self.ptr) };
}
}
}
+125
View File
@@ -0,0 +1,125 @@
use std::mem;
// AsRawFd is required by `frame.fd.as_raw_fd()` below but rustc emits a false
// "unused_imports" warning because OwnedFd also has an inherent `as_raw_fd`.
// E0599 if removed -> must stay; warning is a known rustc quirk.
use std::os::fd::AsRawFd;
use std::os::raw::c_void;
use std::path::Path;
use std::ptr;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use crate::cap_portal::PwDmaBufFrame;
use super::{ff_err, AvHwDevCtx, AvHwFrameCtx};
/// Test whether `drm_device` can import the PipeWire DMA-BUF frame via VAAPI.
pub fn test_dma_buf_import(drm_device: &Path, frame: &PwDmaBufFrame) -> Result<()> {
let hw_dev = AvHwDevCtx::new_vaapi(drm_device)?;
let frames =
AvHwFrameCtx::for_capture(&hw_dev, frame.width, frame.height, ff::format::Pixel::BGRA)?;
// SAFETY: frames is a live VAAPI frames context; frame carries valid DMA-BUF metadata.
unsafe { import_dma_buf_to_vaapi(frames.as_ptr(), frame) }?;
Ok(())
}
/// Import a DMA-BUF into a VAAPI hardware frame via zero-copy `av_hwframe_map`.
///
/// # Safety
/// Imports a DMA-BUF frame into a VAAPI hardware frame pool for GPU-side processing.
///
/// Takes the negotiated format/geometry from `frame` (a `PwDmaBufFrame` from
/// PipeWire capture) plus the target `frames_ctx` (VAAPI frame pool from
/// `AvHwFrameCtx`) and returns an `ff::frame::Video` whose data[3] points to
/// the hardware frame.
///
/// # Safety
///
/// - `frames_ctx` must point to an initialized AVHWCramesContext for VAAPI
/// - `frame.fd` must be a valid DMA-BUF file descriptor
pub unsafe fn import_dma_buf_to_vaapi(
frames_ctx: *mut ffi::AVBufferRef,
frame: &PwDmaBufFrame,
) -> Result<ff::frame::Video> {
let duped_fd = libc::dup(frame.fd.as_raw_fd());
if duped_fd < 0 {
bail!("dup(fd) failed: {}", std::io::Error::last_os_error());
}
let mut desc: ffi::AVDRMFrameDescriptor = mem::zeroed();
desc.nb_objects = 1;
desc.objects[0].fd = duped_fd;
desc.objects[0].size = (frame.height as usize) * (frame.stride as usize);
desc.objects[0].format_modifier = frame.modifier;
desc.nb_layers = 1;
desc.layers[0].format = frame.format;
desc.layers[0].nb_planes = 1;
desc.layers[0].planes[0].object_index = 0;
desc.layers[0].planes[0].offset = frame.offset as isize;
desc.layers[0].planes[0].pitch = frame.stride as isize;
let desc_box = Box::new(desc);
let desc_ptr = Box::into_raw(desc_box);
let buf_ref = ffi::av_buffer_create(
desc_ptr as *mut u8,
std::mem::size_of::<ffi::AVDRMFrameDescriptor>(),
Some(cleanup_drm_descriptor),
ptr::null_mut(),
0,
);
if buf_ref.is_null() {
let desc_box = Box::from_raw(desc_ptr);
libc::close(desc_box.objects[0].fd);
bail!("av_buffer_create returned null for DRM descriptor");
}
let mut src = ff::frame::Video::empty();
{
let sp = src.as_mut_ptr();
(*sp).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME as i32;
(*sp).width = frame.width as i32;
(*sp).height = frame.height as i32;
(*sp).data[0] = (*buf_ref).data;
(*sp).buf[0] = buf_ref;
}
let mut dst = ff::frame::Video::empty();
// SAFETY: frames_ctx is guaranteed by this unsafe function's contract to be a
// valid initialized VAAPI frames context; we set format/hw_frames_ctx on a
// freshly allocated dst frame.
unsafe {
let dp = dst.as_mut_ptr();
(*dp).format = ffi::AVPixelFormat::AV_PIX_FMT_VAAPI as i32;
(*dp).hw_frames_ctx = ffi::av_buffer_ref(frames_ctx);
if (*dp).hw_frames_ctx.is_null() {
bail!("av_buffer_ref(frames_ctx) returned null");
}
}
// SAFETY: src and dst are initialized AVFrames; dst has a valid hw_frames_ctx
// ref and av_hwframe_map fills dst from src.
let ret = unsafe {
ffi::av_hwframe_map(
dst.as_mut_ptr(),
src.as_ptr(),
ffi::AV_HWFRAME_MAP_READ as i32,
)
};
if ret < 0 {
bail!("av_hwframe_map failed: {}", ff_err(ret));
}
Ok(dst)
}
unsafe extern "C" fn cleanup_drm_descriptor(_opaque: *mut c_void, data: *mut u8) {
let desc = data as *mut ffi::AVDRMFrameDescriptor;
if !desc.is_null() && (*desc).nb_objects > 0 && (*desc).objects[0].fd >= 0 {
libc::close((*desc).objects[0].fd);
}
let _ = Box::from_raw(data as *mut ffi::AVDRMFrameDescriptor);
}
+305
View File
@@ -0,0 +1,305 @@
use std::mem;
use std::ptr;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use ffmpeg_next::packet::Mut as _;
use super::encode_output::{self, FrameOutput, PacketOutput};
use super::hash::hash_sampled_y_plane;
use super::{
ff_err, BitrateCommand, CpuNv12Frame, EncodeOutcome, ResolutionChange, SwEncodeTiming,
};
pub struct SwEncEncode {
pub(super) sws_ctx: *mut ffi::SwsContext,
pub(super) enc_video: ff::codec::encoder::video::Video,
pub(super) output: Option<FrameOutput>,
pub(super) yuv_frame: *mut ffi::AVFrame,
pub(super) last_frame_hash: u64,
pub(super) frame_count: u64,
pub(super) starting_timestamp: Option<i64>,
pub(super) frames_written: bool,
pub(super) webrtc_disconnected: bool,
pub(super) webrtc_paused: Option<Arc<AtomicBool>>,
pub(super) bitrate_rx: crossbeam_channel::Receiver<BitrateCommand>,
pub(super) resolution_rx: crossbeam_channel::Receiver<ResolutionChange>,
pub(super) enc_width: u32,
pub(super) enc_height: u32,
pub(super) fps: u32,
pub(super) bitrate: u64,
pub(super) gop_size: u32,
/// Set true when WebRTC requests a keyframe. Forces the next frame to
/// `AV_PICTURE_TYPE_I` and bypasses the dedup hash check. Cleared only
/// after `avcodec_send_frame` accepts the forced frame.
pub(super) force_keyframe_pending: bool,
/// Last per-frame timing snapshot. Reset to `Default` at the start of
/// every `encode_cpu_frame` call (even on early returns) so stale values
/// from a previous frame can never leak out.
pub(super) last_timing: SwEncodeTiming,
/// Capture time of the frame currently being encoded. Saved from the
/// input `CpuNv12Frame` so `drain_encoder` can propagate it into the
/// emitted `EncodedH264Frame` for the frame_age stat (issue #20).
pub(super) last_capture_time: Option<Instant>,
}
/// WebRTC media clock frequency in Hz. Matches RTP clock for video (RFC 3551).
/// Used as encoder time_base denominator for WebRTC mode (1/90000) so that
/// PTS values directly become RTP timestamps with microsecond precision.
/// MP4 mode keeps 1/fps time_base for file output simplicity.
pub const WEBRTC_RTP_CLOCK_HZ: i128 = 90_000;
// SAFETY: SwEncEncode is moved to a single encode thread and accessed only there
// via &mut self. SwsContext, AVFrame, and AVCodecContext are NOT thread-safe for
// concurrent access but are Send-sound under single-thread exclusive use, which
// the encode worker invariant provides. crossbeam Receiver and Arc<AtomicBool>
// are Send by design.
unsafe impl Send for SwEncEncode {}
impl SwEncEncode {
pub fn flush(&mut self) -> Result<()> {
// SAFETY: Sending a null frame flushes the opened software encoder;
// no frame data is dereferenced. enc_video is exclusively borrowed via &mut self.
unsafe {
let ret = ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), ptr::null());
if ret < 0 && ret != ffi::AVERROR_EOF {
bail!("software encoder flush send failed: {}", ff_err(ret));
}
}
let start_ts = self.starting_timestamp.unwrap_or(0);
let _ = self.drain_encoder(start_ts)?;
Ok(())
}
pub fn take_timing(&mut self) -> SwEncodeTiming {
mem::take(&mut self.last_timing)
}
pub fn encode_cpu_frame(&mut self, frame: &CpuNv12Frame) -> Result<EncodeOutcome> {
self.last_timing = SwEncodeTiming::default();
// Save capture_time so drain_encoder can propagate it into the
// EncodedH264Frame emitted via the WebRTC channel (issue #20).
self.last_capture_time = Some(frame.capture_time);
if self.webrtc_disconnected {
return Ok(EncodeOutcome::SkippedDisconnected);
}
// Must drain before the stride check: the import thread emits
// ResolutionChange before the new (smaller-stride) frame arrives.
while let Ok(cmd) = self.bitrate_rx.try_recv() {
match cmd {
BitrateCommand::UpdateBitrate { target_bps } => {
// #23 defensive guardrail: clamp to reasonable max even if policy layer
// is bypassed. 50 Mbps is a hard ceiling; primary cap is enforced in
// state_portal.rs webrtc_thread_loop via --max-bitrate flag.
const ENCODER_BITRATE_HARD_CAP: u64 = 50_000_000;
let target_bps = target_bps.min(ENCODER_BITRATE_HARD_CAP);
tracing::info!(target_bps, "updating encoder bitrate from BWE feedback");
self.bitrate = target_bps;
// SAFETY: enc_video is an opened AVCodecContext exclusively owned by &mut self.
unsafe {
let ctx = self.enc_video.as_mut_ptr();
(*ctx).bit_rate = target_bps as i64;
}
}
BitrateCommand::UpdateResolution { .. } => {}
BitrateCommand::ForceKeyframe => {
self.force_keyframe_pending = true;
tracing::debug!("encode thread: ForceKeyframe requested");
}
}
}
let force_this_frame = self.force_keyframe_pending;
while let Ok(change) = self.resolution_rx.try_recv() {
self.recreate_encoder(change.width, change.height)?;
}
if frame.y_stride < self.enc_width as usize || frame.uv_stride < self.enc_width as usize {
bail!("CPU NV12 frame stride is smaller than encoder width");
}
if let Some(ref paused) = self.webrtc_paused {
if paused.load(Ordering::Relaxed) {
return Ok(EncodeOutcome::SkippedPaused);
}
}
let width = self.enc_width as usize;
let height = self.enc_height as usize;
let required_y_len = frame.y_stride * height.saturating_sub(1) + width;
if frame.y_data.len() < required_y_len {
bail!("CPU NV12 frame Y plane is smaller than encoder dimensions");
}
let frame_index = self.frame_count;
self.frame_count = self.frame_count.saturating_add(1);
let current_hash = hash_sampled_y_plane(&frame.y_data, width, height, frame.y_stride);
let force_gop_frame =
self.gop_size > 0 && frame_index.is_multiple_of(u64::from(self.gop_size));
if frame_index > 0
&& !force_gop_frame
&& !force_this_frame
&& current_hash == self.last_frame_hash
{
tracing::debug!(frame_index, "skipping duplicate frame");
self.last_frame_hash = current_hash;
return Ok(EncodeOutcome::SkippedDuplicate);
}
self.last_frame_hash = current_hash;
let sws_start = Instant::now();
// SAFETY: yuv_frame is an owned reusable YUV420P frame at the same dimensions as sw_nv12;
// sws_ctx was created for NV12 -> YUV420P with no resize, so sws_scale only converts format.
unsafe {
let ret = ffi::av_frame_make_writable(self.yuv_frame);
if ret < 0 {
bail!("av_frame_make_writable failed: {}", ff_err(ret));
}
let src_slices = [
frame.y_data.as_ptr(),
frame.uv_data.as_ptr(),
ptr::null(),
ptr::null(),
];
let src_strides = [frame.y_stride as i32, frame.uv_stride as i32, 0, 0];
let scaled = ffi::sws_scale(
self.sws_ctx,
src_slices.as_ptr(),
src_strides.as_ptr(),
0,
self.enc_height as i32,
(*self.yuv_frame).data.as_ptr() as *mut *mut u8,
(*self.yuv_frame).linesize.as_ptr(),
);
if scaled < 0 {
bail!("sws_scale failed for software encoder: {scaled}");
}
}
let sws_us = sws_start.elapsed().as_micros() as u64;
let pts = frame.pts;
if self.starting_timestamp.is_none() {
self.starting_timestamp = Some(pts);
}
let start_ts = self.starting_timestamp.unwrap_or(0);
let enc_start = Instant::now();
// SAFETY: yuv_frame is initialized, writable, and matches the opened encoder format.
// pict_type is reset every frame: the AVFrame is reused, so without resetting to NONE
// a previously-forced I-type would leak into subsequent P-frames. With forced-idr=1
// set on the encoder, AV_PICTURE_TYPE_I produces a true IDR NALU.
unsafe {
(*self.yuv_frame).pts = pts;
(*self.yuv_frame).pict_type = if force_this_frame {
ffi::AVPictureType::AV_PICTURE_TYPE_I
} else {
ffi::AVPictureType::AV_PICTURE_TYPE_NONE
};
let ret = ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), self.yuv_frame);
if ret < 0 {
bail!(
"avcodec_send_frame failed for software encoder: {}",
ff_err(ret)
);
}
}
if force_this_frame {
self.force_keyframe_pending = false;
}
let output_bytes = self.drain_encoder(start_ts)?;
let encode_us = enc_start.elapsed().as_micros() as u64;
self.last_timing = SwEncodeTiming {
sws_us,
encode_us,
output_bytes,
};
Ok(EncodeOutcome::Encoded)
}
pub(super) fn write_trailer_if_needed(&mut self) -> Result<()> {
if self.frames_written {
if let Some(FrameOutput::Muxer(ref mut octx)) = self.output {
octx.write_trailer()
.map_err(|e| anyhow::anyhow!("Failed to write trailer: {e}"))?;
}
}
Ok(())
}
fn drain_encoder(&mut self, start_ts: i64) -> Result<usize> {
let mut total_bytes = 0usize;
loop {
let mut pkt = ff::Packet::empty();
// SAFETY: enc_video is an open encoder; pkt is writable packet storage.
let ret = unsafe {
ffi::avcodec_receive_packet(self.enc_video.as_mut_ptr(), pkt.as_mut_ptr())
};
if ret < 0 {
if ret == ffi::AVERROR(ffi::EAGAIN) || ret == ffi::AVERROR_EOF {
break;
}
bail!("avcodec_receive_packet failed: {}", ff_err(ret));
}
// Count encoded bytes produced before the Muxer/Channel match to
// avoid branch duplication and handle multi-packet drain correctly.
// SAFETY: pkt was just filled by a successful avcodec_receive_packet;
// the size field is valid and initialized.
let pkt_size = unsafe { (*pkt.as_mut_ptr()).size };
if pkt_size > 0 {
total_bytes += pkt_size as usize;
}
match self.output {
Some(FrameOutput::Muxer(ref mut octx)) => {
encode_output::write_muxer_packet(
&mut pkt,
octx,
self.enc_video.time_base(),
start_ts,
)?;
self.frames_written = true;
}
Some(FrameOutput::Channel(ref tx))
if encode_output::send_channel_packet(
&mut pkt,
tx,
start_ts,
self.last_capture_time.unwrap_or_else(Instant::now),
)? == PacketOutput::Disconnected =>
{
self.webrtc_disconnected = true;
break;
}
Some(FrameOutput::Channel(_)) => {}
None => {}
}
}
Ok(total_bytes)
}
}
impl Drop for SwEncEncode {
fn drop(&mut self) {
if !self.sws_ctx.is_null() {
// SAFETY: sws_ctx is owned by this state and was returned by sws_getContext.
unsafe { ffi::sws_freeContext(self.sws_ctx) };
self.sws_ctx = ptr::null_mut();
}
if !self.yuv_frame.is_null() {
// SAFETY: yuv_frame is owned by this state and was allocated by av_frame_alloc.
unsafe { ffi::av_frame_free(&mut self.yuv_frame) };
}
}
}
+133
View File
@@ -0,0 +1,133 @@
use std::path::Path;
use std::ptr;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use anyhow::Result;
use ffmpeg_next::ffi;
use super::encode_output::FrameOutput;
use super::software::{
alloc_yuv420p_frame, create_nv12_to_yuv420p_sws, create_software_h264_encoder,
create_software_h264_muxer,
};
use super::{BitrateCommand, EncodedH264Frame, ResolutionChange, SwEncEncode, SwEncodeTiming};
impl SwEncEncode {
#[allow(clippy::too_many_arguments)]
pub(super) fn new_muxer(
output_path: &Path,
enc_width: u32,
enc_height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
) -> Result<Self> {
let sws_ctx = create_nv12_to_yuv420p_sws(enc_width, enc_height)?;
let (enc_video, octx) =
create_software_h264_muxer(output_path, enc_width, enc_height, fps, bitrate, gop_size)?;
let yuv_frame = alloc_yuv420p_frame(enc_width, enc_height)?;
let (dummy_tx, bitrate_rx) = crossbeam_channel::bounded(1);
drop(dummy_tx);
let (dummy_resolution_tx, resolution_rx) = crossbeam_channel::bounded(1);
drop(dummy_resolution_tx);
Ok(Self {
sws_ctx,
enc_video,
output: Some(FrameOutput::Muxer(octx)),
yuv_frame,
last_frame_hash: 0,
frame_count: 0,
starting_timestamp: None,
frames_written: false,
webrtc_disconnected: false,
webrtc_paused: None,
bitrate_rx,
resolution_rx,
enc_width,
enc_height,
fps,
bitrate,
gop_size,
force_keyframe_pending: false,
last_timing: SwEncodeTiming::default(),
last_capture_time: None,
})
}
#[allow(clippy::too_many_arguments)]
pub fn new_webrtc(
enc_width: u32,
enc_height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
tx: crossbeam_channel::Sender<EncodedH264Frame>,
webrtc_paused: Arc<AtomicBool>,
bitrate_rx: crossbeam_channel::Receiver<BitrateCommand>,
resolution_rx: crossbeam_channel::Receiver<ResolutionChange>,
) -> Result<Self> {
let sws_ctx = create_nv12_to_yuv420p_sws(enc_width, enc_height)?;
let enc_video =
create_software_h264_encoder(enc_width, enc_height, fps, bitrate, gop_size)?;
let yuv_frame = alloc_yuv420p_frame(enc_width, enc_height)?;
Ok(Self {
sws_ctx,
enc_video,
output: Some(FrameOutput::Channel(tx)),
yuv_frame,
last_frame_hash: 0,
frame_count: 0,
starting_timestamp: None,
frames_written: false,
webrtc_disconnected: false,
webrtc_paused: Some(webrtc_paused),
bitrate_rx,
resolution_rx,
enc_width,
enc_height,
fps,
bitrate,
gop_size,
force_keyframe_pending: false,
last_timing: SwEncodeTiming::default(),
last_capture_time: None,
})
}
pub(super) fn recreate_encoder(&mut self, width: u32, height: u32) -> Result<()> {
if width == self.enc_width && height == self.enc_height {
return Ok(());
}
tracing::info!(
from = format_args!("{}x{}", self.enc_width, self.enc_height),
to = format_args!("{}x{}", width, height),
"recreating WebRTC software encoder for resolution change"
);
if !self.sws_ctx.is_null() {
// SAFETY: sws_ctx is owned exclusively by self and will be replaced below.
unsafe { ffi::sws_freeContext(self.sws_ctx) };
self.sws_ctx = ptr::null_mut();
}
if !self.yuv_frame.is_null() {
// SAFETY: yuv_frame is owned exclusively by self and will be replaced below.
unsafe { ffi::av_frame_free(&mut self.yuv_frame) };
}
self.sws_ctx = create_nv12_to_yuv420p_sws(width, height)?;
self.enc_video =
create_software_h264_encoder(width, height, self.fps, self.bitrate, self.gop_size)?;
self.yuv_frame = alloc_yuv420p_frame(width, height)?;
self.enc_width = width;
self.enc_height = height;
self.last_frame_hash = 0;
self.frame_count = 0;
self.force_keyframe_pending = true;
Ok(())
}
}
+99
View File
@@ -0,0 +1,99 @@
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::packet::Mut as _;
use super::EncodedH264Frame;
pub enum FrameOutput {
Muxer(ff::format::context::Output),
Channel(crossbeam_channel::Sender<EncodedH264Frame>),
}
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub(super) enum PacketOutput {
Written,
Dropped,
Disconnected,
}
pub(super) fn write_muxer_packet(
pkt: &mut ff::Packet,
octx: &mut ff::format::context::Output,
enc_tb: ff::Rational,
start_ts: i64,
) -> Result<()> {
// SAFETY: muxer output was created with stream 0 during setup; streams
// is non-null and stream 0 remains owned by the format context.
let stream_tb = unsafe {
let fmt = *octx.as_ptr();
if fmt.nb_streams == 0 || fmt.streams.is_null() {
bail!("no streams in output context");
}
let st = *fmt.streams.add(0);
ff::Rational::from((*st).time_base)
};
pkt.rescale_ts(enc_tb, stream_tb);
if let Some(pts) = pkt.pts() {
pkt.set_pts(Some(pts - start_ts));
}
if let Some(dts) = pkt.dts() {
pkt.set_dts(Some(dts - start_ts));
}
pkt.set_stream(0);
pkt.write_interleaved(octx)
.map_err(|e| anyhow::anyhow!("Failed to write packet: {e}"))
}
pub(super) fn send_channel_packet(
pkt: &mut ff::Packet,
tx: &crossbeam_channel::Sender<EncodedH264Frame>,
start_ts: i64,
capture_time: Instant,
) -> Result<PacketOutput> {
// SAFETY: pkt is a valid AVPacket just filled by avcodec_receive_packet;
// this copies fields for read-only inspection before pkt is dropped.
let raw = unsafe { *pkt.as_mut_ptr() };
if raw.size <= 0 || raw.data.is_null() {
return Ok(PacketOutput::Dropped);
}
// SAFETY: `pkt` is a valid AVPacket just filled by a successful
// `avcodec_receive_packet` call. We checked `size > 0` and `data` is
// non-null, so `data` points to `size` initialized bytes owned by the
// packet. `u8` has alignment 1, and the slice is copied into a Vec before
// the packet is unreffed.
let data = unsafe { std::slice::from_raw_parts(raw.data, raw.size as usize) };
let pts_ticks = match pkt.pts() {
Some(p) => p - start_ts,
None => {
tracing::warn!("encoder produced packet without PTS, dropping");
return Ok(PacketOutput::Dropped);
}
};
match tx.try_send(EncodedH264Frame {
data: data.to_vec(),
pts_ticks,
capture_time,
}) {
Ok(()) => Ok(PacketOutput::Written),
Err(crossbeam_channel::TrySendError::Full(frame)) => {
tracing::warn!(
"WebRTC channel full, dropping frame: {} bytes lost",
frame.data.len()
);
Ok(PacketOutput::Dropped)
}
Err(crossbeam_channel::TrySendError::Disconnected(frame)) => {
tracing::warn!(
"WebRTC channel disconnected: {} bytes lost",
frame.data.len()
);
Ok(PacketOutput::Disconnected)
}
}
}
+174
View File
@@ -0,0 +1,174 @@
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use crate::transform::Transform;
use super::{ff_err, AvHwDevCtx, AvHwFrameCtx};
#[allow(clippy::too_many_arguments)]
pub(super) fn build_swenc_filter_graph(
hw_dev: &AvHwDevCtx,
frames_rgb: &AvHwFrameCtx,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
) -> Result<ff::filter::Graph> {
let mut graph = ff::filter::Graph::new();
let buffersrc =
ff::filter::find("buffer").ok_or_else(|| anyhow::anyhow!("filter 'buffer' not found"))?;
let buffersink = ff::filter::find("buffersink")
.ok_or_else(|| anyhow::anyhow!("filter 'buffersink' not found"))?;
let scale_vaapi = ff::filter::find("scale_vaapi")
.ok_or_else(|| anyhow::anyhow!("filter 'scale_vaapi' not found"))?;
// FFmpeg 8.0+ rejects VAAPI pix_fmt in buffer args before hw_frames_ctx is attached.
// Use a SW placeholder, then override format/hw_frames_ctx with av_buffersrc_parameters_set.
let args = format!(
"video_size={}x{}:pix_fmt=bgra:time_base=1/{fps}:pixel_aspect=1/1",
width, height,
);
let mut src_ctx = graph.add(&buffersrc, "in", &args)?;
// SAFETY: av_buffersrc_parameters_alloc returns newly allocated parameters
// or null, which is checked below.
let par = unsafe { ffi::av_buffersrc_parameters_alloc() };
if par.is_null() {
bail!("av_buffersrc_parameters_alloc returned null");
}
// SAFETY: par and src_ctx are valid; frames_rgb.ref_clone returns an owned hw_frames_ctx ref
// that buffersrc consumes on successful parameter set.
unsafe {
(*par).format = Into::<ffi::AVPixelFormat>::into(ff::format::Pixel::VAAPI) as i32;
(*par).width = width as i32;
(*par).height = height as i32;
(*par).time_base = ffi::AVRational {
num: 1,
den: fps as i32,
};
(*par).hw_frames_ctx = frames_rgb.ref_clone();
let ret = ffi::av_buffersrc_parameters_set(src_ctx.as_mut_ptr(), par);
ffi::av_free(par as *mut _);
if ret < 0 {
bail!("av_buffersrc_parameters_set failed: {}", ff_err(ret));
}
}
let mut scale_ctx = graph.add(
&scale_vaapi,
"scale",
&format!("{enc_width}:{enc_height}:format=nv12"),
)?;
// SAFETY: scale_vaapi keeps a ref-counted device context while the graph is alive.
unsafe {
(*scale_ctx.as_mut_ptr()).hw_device_ctx = hw_dev.ref_clone();
}
let mut sink_ctx = graph.add(&buffersink, "out", "")?;
src_ctx.link(0, &mut scale_ctx, 0);
scale_ctx.link(0, &mut sink_ctx, 0);
graph
.validate()
.map_err(|e| anyhow::anyhow!("software GPU filter graph validation failed: {e}"))?;
Ok(graph)
}
pub(super) fn build_filter_graph(
hw_dev: &AvHwDevCtx,
frames_rgb: &AvHwFrameCtx,
width: u32,
height: u32,
fps: u32,
transform: Transform,
) -> Result<ff::filter::Graph> {
let mut graph = ff::filter::Graph::new();
let buffersrc =
ff::filter::find("buffer").ok_or_else(|| anyhow::anyhow!("filter 'buffer' not found"))?;
let buffersink = ff::filter::find("buffersink")
.ok_or_else(|| anyhow::anyhow!("filter 'buffersink' not found"))?;
let scale_vaapi = ff::filter::find("scale_vaapi")
.ok_or_else(|| anyhow::anyhow!("filter 'scale_vaapi' not found"))?;
// buffersrc - use AVBufferSrcParameters to set hw_frames_ctx properly.
let args = format!(
"video_size={}x{}:pix_fmt={}:time_base=1/{fps}:pixel_aspect=1/1",
width,
height,
Into::<ffi::AVPixelFormat>::into(ff::format::Pixel::VAAPI) as i32,
);
let mut src_ctx = graph.add(&buffersrc, "in", &args)?;
// SAFETY: av_buffersrc_parameters_alloc allocates params for the buffersrc.
let par = unsafe { ffi::av_buffersrc_parameters_alloc() };
if par.is_null() {
bail!("av_buffersrc_parameters_alloc returned null");
}
// SAFETY: Set hw_frames_ctx on the buffersrc parameters, then apply.
unsafe {
(*par).format = Into::<ffi::AVPixelFormat>::into(ff::format::Pixel::VAAPI) as i32;
(*par).width = width as i32;
(*par).height = height as i32;
(*par).time_base = ffi::AVRational {
num: 1,
den: fps as i32,
};
(*par).hw_frames_ctx = frames_rgb.ref_clone();
let ret = ffi::av_buffersrc_parameters_set(src_ctx.as_mut_ptr(), par);
ffi::av_free(par as *mut _);
if ret < 0 {
bail!("av_buffersrc_parameters_set failed: {}", ff_err(ret));
}
}
// scale_vaapi: hardware scaling and colourspace conversion (keeps original dimensions).
let mut scale_ctx = graph.add(
&scale_vaapi,
"scale",
&format!("{width}:{height}:format=nv12"),
)?;
// SAFETY: scale_vaapi needs hw_device_ctx for VAAPI device access.
unsafe {
(*scale_ctx.as_mut_ptr()).hw_device_ctx = hw_dev.ref_clone();
}
let mut sink_ctx = graph.add(&buffersink, "out", "")?;
src_ctx.link(0, &mut scale_ctx, 0);
match transform {
Transform::Normal => {
scale_ctx.link(0, &mut sink_ctx, 0);
}
other => {
let transpose = ff::filter::find("transpose_vaapi")
.ok_or_else(|| anyhow::anyhow!("filter 'transpose_vaapi' not found"))?;
let dir_val = match other {
Transform::Normal90 => "1",
Transform::Normal180 => "4",
Transform::Normal270 => "2",
Transform::Flipped => "5",
Transform::Flipped90 => "3",
Transform::Flipped180 => "6",
Transform::Flipped270 => "0",
Transform::Normal => unreachable!(),
};
let mut trans_ctx = graph.add(&transpose, "transpose", &format!("dir={dir_val}"))?;
// SAFETY: trans_ctx is a live transpose_vaapi filter context;
// scale_vaapi/transpose_vaapi keep a ref-counted device context.
unsafe {
(*trans_ctx.as_mut_ptr()).hw_device_ctx = hw_dev.ref_clone();
}
scale_ctx.link(0, &mut trans_ctx, 0);
trans_ctx.link(0, &mut sink_ctx, 0);
}
}
graph
.validate()
.map_err(|e| anyhow::anyhow!("Filter graph validation failed: {e}"))?;
Ok(graph)
}
+285
View File
@@ -0,0 +1,285 @@
use std::path::Path;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use ffmpeg_next::packet::Mut as _;
use crate::transform::Transform;
use super::{
ff_err, filter::build_filter_graph, hardware_encoder, hardware_muxer, AvHwDevCtx, AvHwFrameCtx,
EncodeStages,
};
pub struct EncState {
enc_video: ff::codec::encoder::video::Video,
frames_rgb: AvHwFrameCtx,
video_filter: ff::filter::Graph,
// Root AVHWDeviceContext, kept for ownership. Each consumer (encoder,
// filter graph, frames ctx) already holds its own ref_clone(); this
// field is never read after `new()` but must outlive those clones.
#[allow(dead_code)]
hw_device_ctx: AvHwDevCtx,
octx: ff::format::context::Output,
starting_timestamp: Option<i64>,
frames_written: bool,
}
// SAFETY: EncState is moved to exactly one encode worker thread and all Rust
// methods take &mut self, so there is no concurrent *Rust-side* access.
// FFmpeg-internal codec threads may touch hw_device_ctx / frames_rgb through
// the encoder context if frame/slice threading is enabled; this is sound
// because AVHWDeviceContext and AVHWFramesContext are designed for such
// sharing (atomic refcounts, thread-safe pool, libva VADisplay thread safety).
// This impl only lifts auto-Send inference through raw pointers inside the
// ffmpeg-next wrappers; it does not introduce new sharing. Do NOT add fields
// that create shared mutable state across threads without re-auditing.
unsafe impl Send for EncState {}
impl EncState {
#[allow(clippy::too_many_arguments)]
pub fn new(
drm_device: &Path,
output_path: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
bitrate: u64,
gop_size: u32,
fps: u32,
transform: Transform,
existing_hw_ctx: Option<AvHwDevCtx>,
) -> Result<Self> {
tracing::info!(
"EncState::new: {width}x{height} enc={enc_width}x{enc_height} transform={transform:?}"
);
let hw_device_ctx = match existing_hw_ctx {
Some(ctx) => ctx,
None => AvHwDevCtx::new_vaapi(drm_device)?,
};
let frames_rgb =
AvHwFrameCtx::for_capture(&hw_device_ctx, width, height, ff::format::Pixel::BGRA)?;
let mut video_filter =
build_filter_graph(&hw_device_ctx, &frames_rgb, width, height, fps, transform)?;
let mut sink_ctx = video_filter
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
// SAFETY: sink_ctx is a live buffersink; the returned hw_frames_ctx is
// borrowed, so av_buffer_ref creates an owned reference.
let sink_hw_frames = unsafe {
let raw = ffi::av_buffersink_get_hw_frames_ctx(sink_ctx.as_mut_ptr());
if raw.is_null() {
bail!("buffersink has no hw_frames_ctx - filter graph may not be configured for hardware output");
}
let hw_ref = ffi::av_buffer_ref(raw);
if hw_ref.is_null() {
bail!("av_buffer_ref failed for buffersink hw_frames_ctx - likely out of memory");
}
hw_ref
};
// SAFETY: sink_hw_frames is an owned AVBufferRef to an AVHWFramesContext
// returned by the validated filter graph.
unsafe {
let fc = (*sink_hw_frames).data as *mut ffi::AVHWFramesContext;
let actual_w = (*fc).width as u32;
let actual_h = (*fc).height as u32;
if actual_w != enc_width || actual_h != enc_height {
tracing::warn!(
"Filter output dimensions {actual_w}x{actual_h} differ from encoder dimensions {enc_width}x{enc_height}"
);
}
}
let enc_video = hardware_encoder::open_h264_vaapi_encoder(
&hw_device_ctx,
sink_hw_frames,
enc_width,
enc_height,
bitrate,
gop_size,
fps,
)?;
let octx = hardware_muxer::create_output_context(output_path, &enc_video)?;
Ok(Self {
enc_video,
frames_rgb,
video_filter,
hw_device_ctx,
octx,
starting_timestamp: None,
frames_written: false,
})
}
pub fn frames_rgb(&self) -> &AvHwFrameCtx {
&self.frames_rgb
}
pub fn encode_frame(&mut self, hw_frame: &ff::frame::Video) -> Result<EncodeStages> {
let mut filter_src_ctx = self
.video_filter
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source();
let mut filter_sink_ctx = self
.video_filter
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink();
// Scale stage = filter graph push + pull (scale_vaapi for resolution
// change + format conversion to NV12). Timed separately from the
// actual avcodec_send_frame so the per-stage stats answer "where is
// latency?" honestly. See Oracle audit 2026-06-28 step 4.
let scale_start = Instant::now();
filter_src
.add(hw_frame)
.map_err(|e| anyhow::anyhow!("Filter source add failed: {e}"))?;
let mut scale_us = 0u64;
let mut encode_us = 0u64;
loop {
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => {
if filtered.pts().is_none() {
filtered.set_pts(hw_frame.pts());
}
}
Err(ff::Error::Other { errno }) if errno == ffi::EAGAIN => break,
Err(e) => bail!("Filter sink get frame failed: {e}"),
}
// First successful pull closes the scale-stage measurement; later
// pulls (rare extras) roll into encode time.
if scale_us == 0 {
scale_us = scale_start.elapsed().as_micros() as u64;
}
let pts = filtered.pts().unwrap_or(0);
if self.starting_timestamp.is_none() {
self.starting_timestamp = Some(pts);
}
let start_ts = self.starting_timestamp.unwrap();
let encode_start = Instant::now();
// SAFETY: avcodec_send_frame sends a valid NV12 VAAPI surface to the encoder.
let ret =
unsafe { ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), filtered.as_ptr()) };
if ret < 0 {
bail!("avcodec_send_frame failed: {}", ff_err(ret));
}
self.drain_encoder(start_ts)?;
encode_us += encode_start.elapsed().as_micros() as u64;
}
Ok(EncodeStages {
scale_us,
// HW path stays on GPU - no CPU readback, transfer is N/A.
transfer_us: 0,
encode_us,
})
}
pub fn flush(&mut self) -> Result<()> {
let mut filter_src_ctx = self
.video_filter
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source();
if let Err(e) = filter_src.flush() {
tracing::debug!("filter source flush error: {e}");
}
let mut filter_sink_ctx = self
.video_filter
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink();
loop {
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => {
let start_ts = self.starting_timestamp.unwrap_or(0);
// SAFETY: filtered is a valid VAAPI frame drained from the
// filter graph; enc_video is an opened encoder.
let ret = unsafe {
ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), filtered.as_ptr())
};
if ret < 0 {
bail!("avcodec_send_frame failed during flush: {}", ff_err(ret));
}
self.drain_encoder(start_ts)?;
}
Err(_) => break,
}
}
// SAFETY: Sending null frame signals end of stream to encoder.
unsafe {
ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), std::ptr::null());
}
let start_ts = self.starting_timestamp.unwrap_or(0);
self.drain_encoder(start_ts)?;
if self.frames_written {
self.octx
.write_trailer()
.map_err(|e| anyhow::anyhow!("Failed to write trailer: {e}"))?;
}
Ok(())
}
fn drain_encoder(&mut self, start_ts: i64) -> Result<()> {
loop {
let mut pkt = ff::Packet::empty();
// SAFETY: avcodec_receive_packet retrieves an encoded packet.
let ret = unsafe {
ffi::avcodec_receive_packet(self.enc_video.as_mut_ptr(), pkt.as_mut_ptr())
};
if ret < 0 {
if ret == ffi::AVERROR(ffi::EAGAIN) || ret == ffi::AVERROR_EOF {
break;
}
bail!("avcodec_receive_packet failed: {}", ff_err(ret));
}
let enc_tb = self.enc_video.time_base();
// SAFETY: octx was created with stream 0 during muxer setup; streams
// is non-null and stream 0 remains owned by the format context.
let stream_tb = unsafe {
let fmt = *self.octx.as_ptr();
if fmt.nb_streams == 0 || fmt.streams.is_null() {
bail!("no streams in output context");
}
let st = *fmt.streams.add(0);
ff::Rational::from((*st).time_base)
};
pkt.rescale_ts(enc_tb, stream_tb);
if let Some(pts) = pkt.pts() {
pkt.set_pts(Some(pts - start_ts));
}
if let Some(dts) = pkt.dts() {
pkt.set_dts(Some(dts - start_ts));
}
pkt.set_stream(0);
pkt.write_interleaved(&mut self.octx)
.map_err(|e| anyhow::anyhow!("Failed to write packet: {e}"))?;
self.frames_written = true;
}
Ok(())
}
}
+83
View File
@@ -0,0 +1,83 @@
use std::ffi::CString;
use anyhow::Result;
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::{ff_err, AvHwDevCtx};
#[allow(clippy::too_many_arguments)]
pub(super) fn open_h264_vaapi_encoder(
hw_device_ctx: &AvHwDevCtx,
sink_hw_frames: *mut ffi::AVBufferRef,
enc_width: u32,
enc_height: u32,
bitrate: u64,
gop_size: u32,
fps: u32,
) -> Result<ff::codec::encoder::video::Video> {
let codec = ff::encoder::find_by_name("h264_vaapi")
.ok_or_else(|| anyhow::anyhow!("h264_vaapi encoder not found"))?;
let mut enc = {
let ctx = ff::codec::Context::new_with_codec(codec);
ctx.encoder().video()?
};
enc.set_width(enc_width);
enc.set_height(enc_height);
enc.set_format(ff::format::Pixel::VAAPI);
enc.set_bit_rate(bitrate as usize);
enc.set_gop(gop_size);
enc.set_time_base(ff::Rational::new(1, fps as i32));
enc.set_max_b_frames(0);
// VBV rate limiting: caps IDR burst size for WebRTC. Without this a 4K
// scene change can produce a 256KB keyframe that overflows the UDP send
// buffer. bufsize=bitrate/4 is about 250ms of video at the target bitrate.
// SAFETY: enc.as_mut_ptr() is a valid AVCodecContext for the not-yet-opened
// encoder. rc_max_rate and rc_buffer_size are plain integer fields; assigning
// i64/i32 values is a simple struct-field write on a properly aligned pointer.
unsafe {
let ctx_ptr = enc.as_mut_ptr();
(*ctx_ptr).rc_max_rate = bitrate as i64;
(*ctx_ptr).rc_buffer_size = (bitrate / 4) as i32;
}
// SAFETY: AV_CODEC_FLAG_GLOBAL_HEADER must be set BEFORE opening the encoder.
// It triggers SPS/PPS extradata generation needed by the muxer for
// Annex B to AVCC conversion.
unsafe {
(*enc.as_mut_ptr()).flags |= ffi::AV_CODEC_FLAG_GLOBAL_HEADER as i32;
}
// SAFETY: Assign hw device and frames ctx to the encoder.
unsafe {
(*enc.as_mut_ptr()).hw_device_ctx = hw_device_ctx.ref_clone();
(*enc.as_mut_ptr()).hw_frames_ctx = sink_hw_frames;
}
// SAFETY: Set repeat_pps=1 on the encoder so PPS is inserted in every encoded frame.
// This ensures decoders can start decoding from any frame (important for WebRTC).
// repeat_pps is only available in FFmpeg 7.0+ (not in 6.x). On older
// FFmpeg, IDR frames carry SPS by default; PPS repetition depends on the driver.
// For SPS repetition: IDR frames carry SPS by default, controlled by gop_size/idr_interval.
{
let key = CString::new("repeat_pps").unwrap();
let val = CString::new("1").unwrap();
// SAFETY: enc is a valid AVCodecContext for the not-yet-opened encoder;
// priv_data is the codec's private options struct. key/val are NUL-terminated
// CString that live across the call. av_opt_set is FFmpeg's standard
// option-setter. Failure is non-fatal (returns < 0 on older FFmpeg).
let ret = unsafe {
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0)
};
if ret < 0 {
tracing::warn!("av_opt_set repeat_pps failed ({}), likely FFmpeg < 7.0; continuing without per-frame PPS", ff_err(ret));
}
}
let opened = enc
.open()
.map_err(|e| anyhow::anyhow!("Failed to open h264_vaapi encoder: {e}"))?;
Ok(opened.0)
}
+92
View File
@@ -0,0 +1,92 @@
use std::ffi::CString;
use std::path::Path;
use std::ptr;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::ff_err;
pub(super) fn create_output_context(
output_path: &Path,
enc_video: &ff::codec::encoder::video::Video,
) -> Result<ff::format::context::Output> {
let output_cstr = CString::new(output_path.to_str().unwrap())?;
let mut fmt_ctx_ptr: *mut ffi::AVFormatContext = ptr::null_mut();
// SAFETY: avformat_alloc_output_context2 creates format context from
// the file extension. Does NOT open the file.
let ret = unsafe {
ffi::avformat_alloc_output_context2(
&mut fmt_ctx_ptr,
ptr::null_mut(),
ptr::null(),
output_cstr.as_ptr(),
)
};
if ret < 0 || fmt_ctx_ptr.is_null() {
bail!("Failed to allocate output format context: {}", ff_err(ret));
}
// SAFETY: enc_video is a valid AVCodecContext pointer; codec_id is a plain
// i32 enum discriminant read from it. fmt_ctx_ptr is a valid AVFormatContext
// allocated above; oformat is a const pointer field read from it.
// avformat_query_codec checks codec+format compatibility; both pointers are
// valid and FF_COMPLIANCE_NORMAL is a constant. All three reads happen in one
// block so a single SAFETY rationale covers them.
let compat = unsafe {
let codec_id = (*enc_video.as_ptr()).codec_id;
let oformat = (*fmt_ctx_ptr).oformat;
ffi::avformat_query_codec(oformat, codec_id, ffi::FF_COMPLIANCE_NORMAL)
};
if compat < 0 {
bail!("H.264 codec not supported by output container format");
}
// SAFETY: avformat_new_stream creates a new stream in the format context.
let stream_ptr = unsafe { ffi::avformat_new_stream(fmt_ctx_ptr, ptr::null()) };
if stream_ptr.is_null() {
bail!("Failed to create new stream in output context");
}
// SAFETY: avcodec_parameters_from_context copies encoder params + extradata.
let ret =
unsafe { ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr()) };
if ret < 0 {
bail!(
"Failed to copy encoder parameters to stream: {}",
ff_err(ret)
);
}
// SAFETY: Copy encoder time_base to stream.
unsafe {
(*stream_ptr).time_base = (*enc_video.as_ptr()).time_base;
}
// SAFETY: avio_open opens the output file for writing.
let ret = unsafe {
ffi::avio_open(
&mut (*fmt_ctx_ptr).pb,
output_cstr.as_ptr(),
ffi::AVIO_FLAG_WRITE,
)
};
if ret < 0 {
bail!(
"Failed to open output file '{}': {}",
output_path.display(),
ff_err(ret)
);
}
// SAFETY: avformat_write_header writes the container header.
let ret = unsafe { ffi::avformat_write_header(fmt_ctx_ptr, ptr::null_mut()) };
if ret < 0 {
bail!("Failed to write output header: {}", ff_err(ret));
}
// SAFETY: We created fmt_ctx_ptr above and it's valid.
Ok(unsafe { ff::format::context::Output::wrap(fmt_ctx_ptr) })
}
+23
View File
@@ -0,0 +1,23 @@
const FNV1A_OFFSET_BASIS: u64 = 0xcbf29ce484222325;
const FNV1A_PRIME: u64 = 0x100000001b3;
const Y_PLANE_HASH_ROW_STEP: usize = 8;
pub(super) fn hash_sampled_y_plane(
y_data: &[u8],
width: usize,
height: usize,
stride: usize,
) -> u64 {
let mut hash = FNV1A_OFFSET_BASIS;
for row in (0..height).step_by(Y_PLANE_HASH_ROW_STEP) {
let row_start = row * stride;
let row_end = row_start + width;
for &byte in &y_data[row_start..row_end] {
hash ^= u64::from(byte);
hash = hash.wrapping_mul(FNV1A_PRIME);
}
}
hash
}
+263
View File
@@ -0,0 +1,263 @@
use std::path::Path;
use std::slice;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::filter::build_swenc_filter_graph;
use super::{ff_err, AvHwDevCtx, AvHwFrameCtx};
use super::{BitrateCommand, CpuNv12Frame, ResolutionChange};
pub struct SwEncImport {
hw_dev: AvHwDevCtx,
frames_rgb: AvHwFrameCtx,
filter_graph: ff::filter::Graph,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
resolution_rx: Option<crossbeam_channel::Receiver<BitrateCommand>>,
encoder_resolution_tx: Option<crossbeam_channel::Sender<ResolutionChange>>,
}
impl SwEncImport {
#[allow(clippy::too_many_arguments)]
pub fn new(
drm_device: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
) -> Result<Self> {
let hw_dev = AvHwDevCtx::new_vaapi(drm_device)?;
let frames_rgb =
AvHwFrameCtx::for_capture(&hw_dev, width, height, ff::format::Pixel::BGRA)?;
let filter_graph = build_swenc_filter_graph(
&hw_dev,
&frames_rgb,
width,
height,
enc_width,
enc_height,
fps,
)?;
Ok(Self {
hw_dev,
frames_rgb,
filter_graph,
width,
height,
enc_width,
enc_height,
fps,
resolution_rx: None,
encoder_resolution_tx: None,
})
}
#[allow(clippy::too_many_arguments)]
pub fn new_with_resolution_control(
drm_device: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
resolution_rx: crossbeam_channel::Receiver<BitrateCommand>,
encoder_resolution_tx: crossbeam_channel::Sender<ResolutionChange>,
) -> Result<Self> {
let mut this = Self::new(drm_device, width, height, enc_width, enc_height, fps)?;
this.resolution_rx = Some(resolution_rx);
this.encoder_resolution_tx = Some(encoder_resolution_tx);
Ok(this)
}
pub fn frames_rgb(&self) -> &AvHwFrameCtx {
let _ = self.hw_dev.as_ptr();
&self.frames_rgb
}
pub fn import_and_scale(&mut self, hw_frame: &ff::frame::Video) -> Result<CpuNv12Frame> {
self.poll_resolution_commands()?;
let mut filter_src_ctx = self
.filter_graph
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source();
let mut filter_sink_ctx = self
.filter_graph
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink();
filter_src
.add(hw_frame)
.map_err(|e| anyhow::anyhow!("software pipeline filter source add failed: {e}"))?;
let mut first = None;
let mut extra_count = 0usize;
loop {
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => {
if filtered.pts().is_none() {
filtered.set_pts(hw_frame.pts());
}
let cpu_frame = self.transfer_filtered_to_cpu(&filtered)?;
if first.is_none() {
first = Some(cpu_frame);
} else {
extra_count += 1;
}
}
Err(ff::Error::Other { errno }) if errno == ffi::EAGAIN => break,
Err(e) => bail!("software pipeline filter sink get frame failed: {e}"),
}
}
if extra_count > 0 {
tracing::warn!(
"software import filter produced {extra_count} extra frame(s); dropping extras"
);
}
first.ok_or_else(|| anyhow::anyhow!("software pipeline produced no scaled frame"))
}
pub fn flush_import(&mut self) -> Result<Vec<CpuNv12Frame>> {
let mut filter_src_ctx = self
.filter_graph
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source();
if let Err(e) = filter_src.flush() {
tracing::debug!("filter source flush error: {e}");
}
let mut filter_sink_ctx = self
.filter_graph
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink();
let mut frames = Vec::new();
loop {
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => frames.push(self.transfer_filtered_to_cpu(&filtered)?),
Err(_) => break,
}
}
Ok(frames)
}
fn poll_resolution_commands(&mut self) -> Result<()> {
let Some(rx) = self.resolution_rx.as_ref().cloned() else {
return Ok(());
};
let mut requested = None;
while let Ok(cmd) = rx.try_recv() {
match cmd {
BitrateCommand::UpdateResolution { width, height } => {
requested = Some((width & !1, height & !1));
}
BitrateCommand::UpdateBitrate { .. } => {}
BitrateCommand::ForceKeyframe => {}
}
}
let Some((width, height)) = requested else {
return Ok(());
};
if width == self.enc_width && height == self.enc_height {
return Ok(());
}
tracing::info!(
from = format_args!("{}x{}", self.enc_width, self.enc_height),
to = format_args!("{}x{}", width, height),
"rebuilding software import filter graph for resolution change"
);
let _ = self.flush_import();
self.filter_graph = build_swenc_filter_graph(
&self.hw_dev,
&self.frames_rgb,
self.width,
self.height,
width,
height,
self.fps,
)?;
self.enc_width = width;
self.enc_height = height;
if let Some(tx) = &self.encoder_resolution_tx {
tx.send(ResolutionChange { width, height })
.map_err(|_| anyhow::anyhow!("encoder resolution channel disconnected"))?;
}
Ok(())
}
fn transfer_filtered_to_cpu(&self, filtered: &ff::frame::Video) -> Result<CpuNv12Frame> {
// SAFETY: av_frame_alloc returns a newly allocated AVFrame or null,
// which is checked below.
let mut sw_nv12 = unsafe { ffi::av_frame_alloc() };
if sw_nv12.is_null() {
bail!("av_frame_alloc failed for NV12 transfer frame");
}
// SAFETY: sw_nv12 is an allocated destination frame; filtered is a valid VAAPI NV12
// surface produced by scale_vaapi at encoder dimensions.
let transfer_ret = unsafe { ffi::av_hwframe_transfer_data(sw_nv12, filtered.as_ptr(), 0) };
if transfer_ret < 0 {
// SAFETY: sw_nv12 was allocated above and has not been freed yet.
unsafe { ffi::av_frame_free(&mut sw_nv12) };
bail!(
"av_hwframe_transfer_data failed for GPU-downscaled frame: {}",
ff_err(transfer_ret)
);
}
// SAFETY: sw_nv12 was filled by av_hwframe_transfer_data. NV12 planes 0 and 1 are
// initialized for enc_width x enc_height; linesize values define each row's byte span.
let frame = unsafe {
let y_ptr = (*sw_nv12).data[0];
let uv_ptr = (*sw_nv12).data[1];
if y_ptr.is_null() || uv_ptr.is_null() {
ffi::av_frame_free(&mut sw_nv12);
bail!("NV12 transfer frame missing Y/UV plane data");
}
let y_stride = (*sw_nv12).linesize[0] as usize;
let uv_stride = (*sw_nv12).linesize[1] as usize;
if (*sw_nv12).width != self.enc_width as i32
|| (*sw_nv12).height != self.enc_height as i32
{
ffi::av_frame_free(&mut sw_nv12);
bail!("NV12 transfer frame has unexpected dimensions");
}
let y_len = y_stride * self.enc_height as usize;
let uv_len = uv_stride * (self.enc_height as usize / 2);
let y_data = slice::from_raw_parts(y_ptr, y_len).to_vec();
let uv_data = slice::from_raw_parts(uv_ptr, uv_len).to_vec();
let pts = filtered.pts().unwrap_or(0);
ffi::av_frame_free(&mut sw_nv12);
CpuNv12Frame {
y_data,
uv_data,
y_stride,
uv_stride,
pts,
capture_time: std::time::Instant::now(),
}
};
Ok(frame)
}
}
+258
View File
@@ -0,0 +1,258 @@
//! FFmpeg / VAAPI encoder wrappers.
//!
//! ## `Send` justification convention
//!
//! Several types in this module (`AvHwDevCtx`, `AvHwFrameCtx`, `EncState`,
//! `SwEncState`, `SwEncEncode`) carry raw FFmpeg pointers and therefore need
//! an explicit `unsafe impl Send`. The justification is always at the C-API
//! level, never at the Rust-borrow level:
//!
//! - `AVBufferRef` refcounts are `atomic_uint` (`libavutil/buffer.c`), so
//! `av_buffer_ref` / `av_buffer_unref` are safe to call concurrently.
//! - `AVHWDeviceContext` (VAAPI `VADisplay`) is designed by FFmpeg/libva to
//! be shared across codec and filter contexts, including FFmpeg-internal
//! codec worker threads.
//! - `AVHWFramesContext` allocates from an `AVBufferPool` whose get/put are
//! atomic; `av_hwframe_get_buffer` is safe to call concurrently on
//! distinct frames.
//! - `SwsContext`, `AVFilterGraph`, `AVFrame`, `AVCodecContext` are NOT
//! thread-safe for concurrent use, but are `Send`-sound under the
//! single-thread exclusive access invariant that the encode worker
//! enforces.
//!
//! **Anti-pattern**: justifying `Send` with "`&mut self` ensures exclusive
//! access". `Send` is about *moving ownership between threads*, not about
//! borrowing. The `&mut self` on Rust methods is API convenience and is not
//! the basis for soundness — refs cloned via `ref_clone()` routinely escape
//! to other threads / FFmpeg-internal workers, and that is fine because the
//! underlying C APIs are designed for it.
use std::path::Path;
use anyhow::Result;
use crate::transform::{transpose_if_transform_transposed, Transform};
mod device;
mod dmabuf;
mod encode;
mod encode_init;
mod encode_output;
mod filter;
mod hardware;
mod hardware_encoder;
mod hardware_muxer;
mod hash;
mod import;
mod software;
mod state;
mod types;
mod util;
pub use device::{AvHwDevCtx, AvHwFrameCtx};
pub use dmabuf::{import_dma_buf_to_vaapi, test_dma_buf_import};
pub use encode::{SwEncEncode, WEBRTC_RTP_CLOCK_HZ};
#[allow(unused_imports)]
pub use encode_output::FrameOutput;
pub use hardware::EncState;
#[cfg(test)]
use hash::hash_sampled_y_plane;
pub use import::SwEncImport;
pub use state::SwEncState;
pub use types::{
BitrateCommand, CpuNv12Frame, EncodeOutcome, EncodeStages, EncodedH264Frame, ResolutionChange,
SwEncodeTiming,
};
#[allow(unused_imports)]
pub use util::av_err_to_string;
pub(crate) use util::ff_err;
// ---------------------------------------------------------------------------
// Shared encoder creation (used by both wlr-screencopy and portal paths)
// ---------------------------------------------------------------------------
/// Create a fully configured encoder with VAAPI hardware acceleration.
///
/// Convenience wrapper around [`EncState::new`] that computes default values
/// for `bitrate` and `gop_size` when not provided, and handles encoder dimension
/// transposition for rotated/transformed outputs.
#[allow(clippy::too_many_arguments)]
pub fn create_encoder(
drm_device: &Path,
output_path: &Path,
width: u32,
height: u32,
fps: u32,
transform: Transform,
bitrate: Option<u64>,
gop_size: Option<u32>,
existing_hw_ctx: Option<AvHwDevCtx>,
) -> Result<EncState> {
let (enc_w, enc_h) = transpose_if_transform_transposed(transform, width as i32, height as i32);
let actual_bitrate =
bitrate.unwrap_or_else(|| 2 * (width as u64) * (height as u64) * (fps as u64) / 100);
let actual_gop_size = gop_size.unwrap_or(fps);
EncState::new(
drm_device,
output_path,
width,
height,
enc_w as u32,
enc_h as u32,
actual_bitrate,
actual_gop_size,
fps,
transform,
existing_hw_ctx,
)
}
#[cfg(test)]
mod tests {
use super::*;
// Centralizes the `stride * row` byte-offset pattern used by the Y-plane hash
// tests below, so clippy::erasing_op (row == 0) and clippy::identity_op (row == 1)
// both pass without sacrificing the row-index intent the tests are written around.
fn row_range(row: usize, stride: usize, width: usize) -> std::ops::Range<usize> {
let start = stride * row;
start..start + width
}
// ── Task 1: VBV x264opts formatting ──
#[test]
fn vbv_x264opts_format() {
let bitrate: u64 = 5_000_000;
// x264 expects kbit/s and kbit, not bps
let vbv_maxrate_kbps = bitrate / 1000;
let vbv_bufsize_kbps = (bitrate / 4) / 1000;
let opts = format!(
"repeat_headers=1:vbv-maxrate={vbv_maxrate_kbps}:vbv-bufsize={vbv_bufsize_kbps}"
);
assert_eq!(vbv_maxrate_kbps, 5000);
assert_eq!(vbv_bufsize_kbps, 1250);
assert!(opts.contains("vbv-maxrate=5000"));
assert!(opts.contains("vbv-bufsize=1250"));
}
#[test]
fn vbv_bufsize_is_quarter_of_maxrate() {
for bitrate in [1_000_000, 5_000_000, 10_000_000] {
// x264 expects kbit/s and kbit; both scaled by /1000, ratio preserved
let maxrate_kbps = bitrate / 1000;
let bufsize_kbps = (bitrate / 4) / 1000;
assert_eq!(
bufsize_kbps * 4,
maxrate_kbps,
"bufsize should be maxrate/4"
);
}
}
// ── Task 3: GOP formula ──
#[test]
fn webrtc_gop_formula() {
// Formula under test: GOP = max(fps * 2, 20). Hid behind a runtime lambda so
// clippy can't constant-fold the assertions into tautologies (which would
// silently strip the floor-case coverage for 5fps).
fn gop(fps: u32) -> u32 {
(fps * 2).max(20)
}
assert_eq!(gop(15), 30); // 15fps -> 30
assert_eq!(gop(30), 60); // 30fps -> 60
assert_eq!(gop(60), 120); // 60fps -> 120
assert_eq!(gop(5), 20); // 5fps -> 20 (floor)
}
#[test]
fn h264_level_values() {
// Level 4.0 supports up to 1080p@30fps (used for file muxer)
assert_eq!(40i32, 40);
// Level 4.2 supports up to 1440p@30fps (used for WebRTC low-latency encoder)
assert_eq!(42i32, 42);
}
// ── Task 4: Duplicate frame hash detection ──
#[test]
fn hash_sampled_y_plane_first_frame_consistent() {
let width = 64;
let height = 64;
let stride = 64;
let y_data = vec![0u8; stride * height];
let hash1 = hash_sampled_y_plane(&y_data, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data, width, height, stride);
assert_eq!(hash1, hash2, "same input should produce same hash");
}
#[test]
fn hash_sampled_y_plane_detects_different_frames() {
let width = 64;
let height = 64;
let stride = 64;
let y_data1 = vec![0u8; stride * height];
let y_data2 = vec![128u8; stride * height];
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_ne!(
hash1, hash2,
"different frame data should produce different hashes"
);
}
#[test]
fn hash_sampled_y_plane_samples_every_8th_row() {
// Changing a non-sampled row (e.g., row 1) should NOT change the hash
let width = 64;
let height = 64;
let stride = 64;
let y_data1 = vec![0u8; stride * height];
let mut y_data2 = vec![0u8; stride * height];
// Row 1 is NOT sampled (sampling is every 8th row: 0, 8, 16, ...)
y_data2[row_range(1, stride, width)].fill(255);
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_eq!(
hash1, hash2,
"non-sampled row change should not affect hash"
);
}
#[test]
fn hash_sampled_y_plane_sensitive_to_sampled_row() {
// Changing a sampled row (row 0) SHOULD change the hash
let width = 64;
let height = 64;
let stride = 64;
let y_data1 = vec![0u8; stride * height];
let mut y_data2 = vec![0u8; stride * height];
// Row 0 IS sampled (every 8th row starting from 0)
y_data2[row_range(0, stride, width)].fill(255);
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_ne!(
hash1, hash2,
"sampled row change should produce different hash"
);
}
#[test]
fn hash_sampled_y_plane_handles_stride_greater_than_width() {
// Stride can be larger than width due to alignment; unused padding should not affect hash
let width = 32;
let height = 16;
let stride = 64; // padded stride
let y_data1 = vec![0u8; stride * height];
let mut y_data2 = vec![0u8; stride * height];
// Fill the padding area (columns 32..63) of row 0 with garbage
y_data2[width..stride].fill(0xFF);
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_eq!(
hash1, hash2,
"padding bytes beyond width should not affect hash"
);
}
}
+268
View File
@@ -0,0 +1,268 @@
use std::ffi::CString;
use std::path::Path;
use std::ptr;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::ff_err;
pub(super) fn create_nv12_to_yuv420p_sws(width: u32, height: u32) -> Result<*mut ffi::SwsContext> {
// SAFETY: sws_getContext creates an owned scaler context for same-size NV12 -> YUV420P.
let ctx = unsafe {
ffi::sws_getContext(
width as i32,
height as i32,
ffi::AVPixelFormat::AV_PIX_FMT_NV12,
width as i32,
height as i32,
ffi::AVPixelFormat::AV_PIX_FMT_YUV420P,
2,
ptr::null_mut(),
ptr::null_mut(),
ptr::null_mut(),
)
};
if ctx.is_null() {
bail!("Failed to create NV12 -> YUV420P sws_scale context");
}
Ok(ctx)
}
pub(super) fn alloc_yuv420p_frame(width: u32, height: u32) -> Result<*mut ffi::AVFrame> {
// SAFETY: Allocate an AVFrame, configure format/dimensions, then allocate writable buffers.
unsafe {
let mut frame = ffi::av_frame_alloc();
if frame.is_null() {
bail!("av_frame_alloc failed");
}
(*frame).width = width as i32;
(*frame).height = height as i32;
(*frame).format = ffi::AVPixelFormat::AV_PIX_FMT_YUV420P as i32;
let ret = ffi::av_frame_get_buffer(frame, 0);
if ret < 0 {
ffi::av_frame_free(&mut frame);
bail!("av_frame_get_buffer failed: {}", ff_err(ret));
}
Ok(frame)
}
}
pub(super) fn create_software_h264_muxer(
output_path: &Path,
width: u32,
height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
) -> Result<(
ff::codec::encoder::video::Video,
ff::format::context::Output,
)> {
let output_cstr = CString::new(output_path.to_str().unwrap())?;
let codec = ff::encoder::find_by_name("libx264")
.or_else(|| ff::encoder::find_by_name("libopenh264"))
.ok_or_else(|| {
anyhow::anyhow!("No H.264 software encoder found (tried libx264, libopenh264)")
})?;
let codec_name = codec.name().to_string();
let mut enc = {
let ctx = ff::codec::Context::new_with_codec(codec);
ctx.encoder().video()?
};
enc.set_width(width);
enc.set_height(height);
enc.set_format(ff::format::Pixel::YUV420P);
enc.set_bit_rate(bitrate as usize);
enc.set_gop(gop_size);
enc.set_time_base(ff::Rational::new(1, fps as i32));
enc.set_max_b_frames(3);
// SAFETY: global headers are needed by MP4 and harmless for other common muxers.
unsafe {
(*enc.as_mut_ptr()).flags |= ffi::AV_CODEC_FLAG_GLOBAL_HEADER as i32;
}
if codec_name == "libx264" {
// SAFETY: priv_data and codec context belong to the unopened encoder;
// strings live for each av_opt_set call.
unsafe {
let key = CString::new("preset").unwrap();
let val = CString::new("fast").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("threads").unwrap();
let val = CString::new("6").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
(*enc.as_mut_ptr()).profile = ffi::AV_PROFILE_H264_HIGH;
// SAFETY: enc is a valid, initialized AVCodecContext from
// avcodec_alloc_context3. Setting level is a simple i32 field
// assignment on a properly aligned struct.
(*enc.as_mut_ptr()).level = 40; // H.264 Level 4.0 (up to 1080p@30)
}
}
let opened = enc
.open()
.map_err(|e| anyhow::anyhow!("Failed to open {codec_name} encoder: {e}"))?;
let enc_video = opened.0;
let use_null = output_path
.to_str()
.map(|s| s.contains("null"))
.unwrap_or(false);
let fmt_name = if use_null {
CString::new("null").unwrap()
} else {
CString::new("").unwrap()
};
let fmt_name_ptr = if use_null {
fmt_name.as_ptr()
} else {
ptr::null()
};
let mut fmt_ctx_ptr: *mut ffi::AVFormatContext = ptr::null_mut();
// SAFETY: fmt_ctx_ptr is initialized by FFmpeg; C strings live across the call.
let ret = unsafe {
ffi::avformat_alloc_output_context2(
&mut fmt_ctx_ptr,
ptr::null_mut(),
fmt_name_ptr,
output_cstr.as_ptr(),
)
};
if ret < 0 || fmt_ctx_ptr.is_null() {
bail!("Failed to allocate output format context: {}", ff_err(ret));
}
// SAFETY: fmt_ctx_ptr is valid; stream and codec parameters are owned by the format context.
let stream_ptr = unsafe { ffi::avformat_new_stream(fmt_ctx_ptr, ptr::null()) };
if stream_ptr.is_null() {
bail!("Failed to create output stream");
}
// SAFETY: stream_ptr and encoder context are valid; parameters are copied into stream.
let ret =
unsafe { ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr()) };
if ret < 0 {
bail!("Failed to copy codec parameters to stream: {}", ff_err(ret));
}
// SAFETY: stream_ptr is valid and writable during muxer setup.
unsafe {
(*stream_ptr).time_base = (*enc_video.as_ptr()).time_base;
}
// SAFETY: open an AVIO only for muxers that require files; null muxer advertises NOFILE.
unsafe {
if (*(*fmt_ctx_ptr).oformat).flags & ffi::AVFMT_NOFILE == 0 {
let ret = ffi::avio_open(
&mut (*fmt_ctx_ptr).pb,
output_cstr.as_ptr(),
ffi::AVIO_FLAG_WRITE,
);
if ret < 0 {
bail!(
"Failed to open output file '{}': {}",
output_path.display(),
ff_err(ret)
);
}
}
}
// SAFETY: fmt_ctx_ptr is fully configured.
let ret = unsafe { ffi::avformat_write_header(fmt_ctx_ptr, ptr::null_mut()) };
if ret < 0 {
bail!("Failed to write output header: {}", ff_err(ret));
}
// SAFETY: ownership of fmt_ctx_ptr transfers to ffmpeg-next Output wrapper.
let octx = unsafe { ff::format::context::Output::wrap(fmt_ctx_ptr) };
tracing::info!("Using software H.264 encoder: {codec_name}");
Ok((enc_video, octx))
}
pub(super) fn create_software_h264_encoder(
width: u32,
height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
) -> Result<ff::codec::encoder::video::Video> {
let codec = ff::encoder::find_by_name("libx264")
.or_else(|| ff::encoder::find_by_name("libopenh264"))
.ok_or_else(|| anyhow::anyhow!("No H.264 software encoder found"))?;
let codec_name = codec.name().to_string();
let mut enc = {
let ctx = ff::codec::Context::new_with_codec(codec);
ctx.encoder().video()?
};
enc.set_width(width);
enc.set_height(height);
enc.set_format(ff::format::Pixel::YUV420P);
enc.set_bit_rate(bitrate as usize);
enc.set_gop(gop_size);
// 90kHz media clock matches RTP directly. Eliminates 1/fps quantization
// that previously caused sequential RTP timestamps during 60fps capture,
// leading to 2x RTP time inflation and 10s+ browser jitter buffer growth.
// See issue #25.
enc.set_time_base(ff::Rational::new(1, 90_000));
// Explicit framerate is REQUIRED when time_base is not 1/fps, otherwise
// libx264 infers wrong fps from the 90kHz time_base and VBV rate control
// breaks. Per Oracle review round for #25.
enc.set_frame_rate(Some(ff::Rational::new(fps as i32, 1)));
enc.set_max_b_frames(0);
if codec_name == "libx264" {
// SAFETY: priv_data and codec context belong to the unopened encoder;
// each CString lives for the duration of its av_opt_set call.
unsafe {
let key = CString::new("preset").unwrap();
let val = CString::new("veryfast").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("tune").unwrap();
let val = CString::new("zerolatency").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("threads").unwrap();
let val = CString::new("6").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
// High profile via AVCodecContext.profile (not x264opts - x264 rejects it there).
// High enables CABAC + 8x8dct automatically.
(*enc.as_mut_ptr()).profile = ffi::AV_PROFILE_H264_HIGH;
// SAFETY: enc is a valid, initialized AVCodecContext from
// avcodec_alloc_context3. Setting level is a simple i32 field
// assignment on a properly aligned struct.
(*enc.as_mut_ptr()).level = 42; // H.264 Level 4.2 (up to 1440p@30)
// SAFETY: priv_data belongs to the unopened libx264 encoder context.
// `forced-idr` is an FFmpeg-level private option (not x264-native),
// so it must be set via av_opt_set, NOT via the x264opts string.
// With forced-idr=1, setting AV_PICTURE_TYPE_I on an input frame
// produces a true IDR NALU with inline SPS/PPS (repeat_headers=1).
let key = CString::new("forced-idr").unwrap();
let val = CString::new("1").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("x264opts").unwrap();
// x264's vbv-maxrate unit is kbit/s and vbv-bufsize is kbit (NOT bps).
// Confirmed via x264 source encoder/ratecontrol.c:658-661 which multiplies
// these values by 1000 to convert kbit -> bit at use site. Passing bps makes
// VBV effectively unbounded (5.5 Mbps becomes 5.5 Gbps, clipped to 2 Gbps).
// See https://github.com/mirror/x264/blob/c24e06c2e184345ceb33eb20a15d1024d9fd3497/encoder/ratecontrol.c#L658-L661
let vbv_maxrate_kbps = bitrate / 1000;
let vbv_bufsize_kbps = (bitrate / 4) / 1000;
let val = CString::new(format!(
"repeat_headers=1:vbv-maxrate={vbv_maxrate_kbps}:vbv-bufsize={vbv_bufsize_kbps}"
))
.unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
}
}
let opened = enc
.open()
.map_err(|e| anyhow::anyhow!("Failed to open {codec_name} encoder: {e}"))?;
tracing::info!("WebRTC encoder: {codec_name} {width}x{height} @ {fps}fps {bitrate}bps (profile High, preset veryfast)");
Ok(opened.0)
}
+111
View File
@@ -0,0 +1,111 @@
use std::path::Path;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use std::time::Instant;
use anyhow::Result;
use ffmpeg_next as ff;
use super::{AvHwFrameCtx, EncodeStages, EncodedH264Frame, SwEncEncode, SwEncImport};
pub struct SwEncState {
import: SwEncImport,
encode: SwEncEncode,
}
// SAFETY: SwEncState is moved to a single encode thread and accessed only there.
// All FFmpeg handles (SwsContext, AVFrame, AVCodecContext) inside SwEncImport /
// SwEncEncode are non-thread-safe but Send-sound under exclusive access.
// Existing sync callers move it across threads only with external serialization.
unsafe impl Send for SwEncState {}
impl SwEncState {
#[allow(clippy::too_many_arguments)]
pub fn new(
drm_device: &Path,
output_path: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
) -> Result<Self> {
tracing::info!(
"SwEncState::new: GPU downscale {width}x{height} BGRA -> {enc_width}x{enc_height} NV12, software H.264"
);
let import = SwEncImport::new(drm_device, width, height, enc_width, enc_height, fps)?;
let encode =
SwEncEncode::new_muxer(output_path, enc_width, enc_height, fps, bitrate, gop_size)?;
Ok(Self { import, encode })
}
#[allow(clippy::too_many_arguments)]
pub fn new_webrtc(
drm_device: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
tx: crossbeam_channel::Sender<EncodedH264Frame>,
webrtc_paused: Arc<AtomicBool>,
) -> Result<Self> {
tracing::info!(
"SwEncState::new_webrtc: GPU downscale {width}x{height} BGRA -> {enc_width}x{enc_height} NV12, software H.264 -> WebRTC"
);
let import = SwEncImport::new(drm_device, width, height, enc_width, enc_height, fps)?;
let (dummy_tx, bitrate_rx) = crossbeam_channel::bounded(1);
drop(dummy_tx);
let (dummy_resolution_tx, resolution_rx) = crossbeam_channel::bounded(1);
drop(dummy_resolution_tx);
let encode = SwEncEncode::new_webrtc(
enc_width,
enc_height,
fps,
bitrate,
gop_size,
tx,
webrtc_paused,
bitrate_rx,
resolution_rx,
)?;
Ok(Self { import, encode })
}
pub fn frames_rgb(&self) -> &AvHwFrameCtx {
self.import.frames_rgb()
}
pub fn encode_frame(&mut self, hw_frame: &ff::frame::Video) -> Result<EncodeStages> {
// SW path: import_and_scale bundles GPU filter graph (scale) + GPU->CPU
// readback (transfer) into one call. Timing them separately requires
// extending import_and_scale's signature; for now both roll into
// scale_us and transfer_us stays 0 with this comment as the honest
// statement. Oracle audit 2026-06-28 step 4.
let scale_start = Instant::now();
let cpu_frame = self.import.import_and_scale(hw_frame)?;
let scale_us = scale_start.elapsed().as_micros() as u64;
let encode_start = Instant::now();
self.encode.encode_cpu_frame(&cpu_frame)?;
let encode_us = encode_start.elapsed().as_micros() as u64;
Ok(EncodeStages {
scale_us,
transfer_us: 0,
encode_us,
})
}
pub fn flush(&mut self) -> Result<()> {
for frame in self.import.flush_import()? {
self.encode.encode_cpu_frame(&frame)?;
}
self.encode.flush()?;
self.encode.write_trailer_if_needed()
}
}
+88
View File
@@ -0,0 +1,88 @@
/// Commands sent from the WebRTC thread to the SW encoder when the
/// bandwidth estimate changes significantly.
pub enum BitrateCommand {
UpdateBitrate {
target_bps: u64,
},
UpdateResolution {
width: u32,
height: u32,
},
/// Force the next encoded frame to be an IDR. Sent by the WebRTC thread
/// in response to str0m `Event::KeyframeRequest` or a resolution change.
ForceKeyframe,
}
#[derive(Clone, Copy, Debug)]
pub struct ResolutionChange {
pub width: u32,
pub height: u32,
}
/// Per-frame timing snapshot for the software encoder, consumed by the stats
/// thread. `sws_us` measures NV12->YUV420P conversion, `encode_us` measures
/// `avcodec_send_frame` + drain, and `output_bytes` counts encoded bytes
/// produced by libavcodec (even if downstream delivery later drops them).
#[derive(Default, Clone, Copy, Debug)]
pub struct SwEncodeTiming {
pub sws_us: u64,
pub encode_us: u64,
pub output_bytes: usize,
}
/// Outcome of a single `encode_cpu_frame` call. Used by the encode thread
/// to decide whether to report timing stats (only real encodes tick encoded_fps).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum EncodeOutcome {
/// Frame was actually encoded and produced output bytes.
Encoded,
/// Frame was dropped because WebRTC is paused (no client connected).
SkippedPaused,
/// Frame was dropped because the encoder is in disconnected state.
SkippedDisconnected,
/// Frame was dropped because its Y-plane hash matched the previous frame.
SkippedDuplicate,
}
/// Per-stage timing breakdown for one encode cycle on the hardware path.
/// Returned by `EncState::encode_frame` so callers can fold the numbers
/// into `crate::stats::FrameTimings`. `transfer_us` is always 0 on the HW
/// path because the frame stays on the GPU; the SW path's struct (if added
/// later) would carry a real readback measurement.
#[derive(Debug, Default, Clone, Copy)]
pub struct EncodeStages {
pub scale_us: u64,
pub transfer_us: u64,
pub encode_us: u64,
}
/// Encoded H.264 frame with timing metadata for WebRTC output.
///
/// MP4 file output (FrameOutput::Muxer) does NOT use this - it writes via
/// avformat which preserves PTS internally. WebRTC output (FrameOutput::Channel)
/// requires explicit PTS propagation so RTP timestamps reflect real capture time.
/// Without this, WebRTC clients' jitter buffers grow to seconds under
/// damage-driven variable frame rate. See issue #24.
#[derive(Debug)]
pub struct EncodedH264Frame {
/// H.264 NAL byte stream (Annex B or AVCC depending on encoder configuration)
pub data: Vec<u8>,
/// PTS in encoder time_base units (1/fps seconds), normalized so first frame = 0.
/// Derived from real capture time, NOT frame counter.
pub pts_ticks: i64,
/// Wall-clock capture time, propagated from CpuNv12Frame for frame_age stat.
pub capture_time: std::time::Instant,
}
/// Owned CPU NV12 frame data for cross-thread transfer.
/// Produced by main thread (VAAPI import + GPU scale + transfer), consumed by encode thread.
pub struct CpuNv12Frame {
pub y_data: Vec<u8>,
pub uv_data: Vec<u8>,
pub y_stride: usize,
pub uv_stride: usize,
pub pts: i64,
/// Wall-clock time when this frame was captured (PipeWire delivery).
/// Used for frame_age stat: time from capture to WebRTC send.
pub capture_time: std::time::Instant,
}
+20
View File
@@ -0,0 +1,20 @@
use ffmpeg_next::ffi;
/// Convert an FFmpeg error code to a human-readable string.
pub fn av_err_to_string(err: i32) -> String {
let mut buf = vec![0u8; 128];
// SAFETY: buf points to 128 writable bytes and lives for the duration of
// av_strerror.
unsafe {
ffi::av_strerror(err, buf.as_mut_ptr() as *mut i8, buf.len());
}
String::from_utf8_lossy(&buf)
.trim_end_matches('\0')
.to_string()
}
/// Format an FFmpeg error code with both numeric value and description.
/// Example output: "error -22 (Invalid argument)"
pub(crate) fn ff_err(ret: i32) -> String {
format!("error {ret} ({})", av_err_to_string(ret))
}
+17 -19
View File
@@ -113,25 +113,23 @@ fn check_portal_available() -> bool {
// The most likely operation to hang — requires actual Portal-side work.
// 最可能卡住的操作,需要 Portal 端实际处理。
let version = match tokio::time::timeout(
PORTAL_DBUS_TIMEOUT,
inner.get_property::<u32>("version"),
)
.await
{
Ok(Ok(version)) => {
tracing::info!("Portal ScreenCast available (version: {version})");
true
}
Ok(Err(e)) => {
tracing::info!("Portal ScreenCast version query failed: {e}");
false
}
Err(_) => {
log_portal_unresponsive("querying ScreenCast version");
false
}
};
let version =
match tokio::time::timeout(PORTAL_DBUS_TIMEOUT, inner.get_property::<u32>("version"))
.await
{
Ok(Ok(version)) => {
tracing::info!("Portal ScreenCast available (version: {version})");
true
}
Ok(Err(e)) => {
tracing::info!("Portal ScreenCast version query failed: {e}");
false
}
Err(_) => {
log_portal_unresponsive("querying ScreenCast version");
false
}
};
version
})
}
+76
View File
@@ -0,0 +1,76 @@
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use ffmpeg_next::packet::Mut;
use wl_webrtc::cap_portal::{CapPortal, PwCtrlEvent};
pub fn receive_first_frame(cap: &CapPortal) -> Result<wl_webrtc::cap_portal::PwDmaBufFrame> {
// Drain-and-wait loop that mirrors production's repeated-poll semantics
// (state_portal.rs::poll_and_encode driven by main.rs's outer loop), but with
// a single bounded 10s total deadline appropriate for a bench tool. Unlike a
// single 10s blocking wait, this loop actually iterates: each turn drains ALL
// pending control events (the ctrl channel is bounded to 8 — a single
// if-let would silently miss backlog) and then waits a short slice for a
// frame, so StreamEnded/Error arriving mid-wait are observed within ~200ms.
const TOTAL_DEADLINE: std::time::Duration = std::time::Duration::from_secs(10);
const WAIT_SLICE: std::time::Duration = std::time::Duration::from_millis(200);
let deadline = Instant::now() + TOTAL_DEADLINE;
loop {
while let Ok(ctrl) = cap.event_receiver().try_recv() {
match ctrl {
PwCtrlEvent::StreamEnded => bail!("PipeWire stream ended before first frame"),
PwCtrlEvent::FormatChanged { .. } => {}
PwCtrlEvent::Error(e) => bail!("PipeWire error: {e}"),
}
}
let remaining = match deadline.checked_duration_since(Instant::now()) {
Some(r) if !r.is_zero() => r,
_ => bail!("Timeout waiting for first frame (10s)"),
};
let slice = remaining.min(WAIT_SLICE);
match cap.frame_receiver().recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(crossbeam_channel::RecvTimeoutError::Timeout) => continue,
Err(crossbeam_channel::RecvTimeoutError::Disconnected) => {
bail!("PipeWire frame channel disconnected");
}
}
}
}
pub fn drain_encoder(
enc_video: &mut ff::encoder::video::Video,
octx: &mut ff::format::context::Output,
) -> Result<()> {
loop {
let mut pkt = ff::Packet::empty();
// SAFETY: enc_video is the opened encoder; pkt is an empty Packet whose
// inner AVPacket pointer is valid. avcodec_receive_packet fills pkt with
// the next encoded packet, or returns EAGAIN/EOF when drained.
let ret = unsafe { ffi::avcodec_receive_packet(enc_video.as_mut_ptr(), pkt.as_mut_ptr()) };
if ret < 0 {
if ret == ffi::AVERROR(ffi::EAGAIN) || ret == ffi::AVERROR_EOF {
break;
}
eprintln!("avcodec_receive_packet failed: {ret}");
break;
}
let enc_tb = enc_video.time_base();
// SAFETY: octx.as_ptr() is a valid AVFormatContext; streams is a NULL-terminated
// array of AVStream*. We index [0] which exists because we created exactly one
// stream in setup. Reading time_base is a plain AVRational field access.
let stream_tb = unsafe {
let streams = (*octx.as_ptr()).streams;
let st = *streams.add(0);
ff::Rational::from((*st).time_base)
};
pkt.rescale_ts(enc_tb, stream_tb);
pkt.set_stream(0);
pkt.write_interleaved(octx)
.map_err(|e| anyhow::anyhow!("write packet failed: {e}"))?;
}
Ok(())
}
@@ -15,75 +15,16 @@ use clap::Parser;
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use ffmpeg_next::packet::Mut;
use wl_webrtc::args::Args;
use wl_webrtc::cap_portal::{CapPortal, PwCtrlEvent};
#[derive(Parser, Debug)]
#[command(
name = "sw_encode_bench",
about = "Software encoding pipeline benchmark"
)]
struct BenchArgs {
#[arg(short, long)]
output: String,
#[path = "../common/mod.rs"]
mod common;
#[arg(long, default_value_t = 120)]
frames: u32,
mod stats;
#[arg(long, default_value_t = 2560)]
enc_width: u32,
#[arg(long, default_value_t = 1440)]
enc_height: u32,
}
#[derive(Default)]
struct FrameStats {
mmap_us: Vec<u64>,
scale_us: Vec<u64>,
encode_us: Vec<u64>,
total_us: Vec<u64>,
mmap_failures: u32,
}
impl FrameStats {
fn avg_ms(data: &[u64]) -> f64 {
if data.is_empty() {
return 0.0;
}
data.iter().sum::<u64>() as f64 / data.len() as f64 / 1000.0
}
}
fn pix_fmt(p: ff::format::Pixel) -> ffi::AVPixelFormat {
Into::<ffi::AVPixelFormat>::into(p)
}
fn receive_first_frame(cap: &CapPortal) -> Result<wl_webrtc::cap_portal::PwDmaBufFrame> {
loop {
if let Ok(ctrl) = cap.event_receiver().try_recv() {
match ctrl {
PwCtrlEvent::StreamEnded => bail!("PipeWire stream ended before first frame"),
PwCtrlEvent::FormatChanged { .. } => {}
PwCtrlEvent::Error(e) => bail!("PipeWire error: {e}"),
}
}
match cap
.frame_receiver()
.recv_timeout(std::time::Duration::from_secs(10))
{
Ok(frame) => return Ok(frame),
Err(crossbeam_channel::RecvTimeoutError::Timeout) => {
bail!("Timeout waiting for first frame (10s)");
}
Err(crossbeam_channel::RecvTimeoutError::Disconnected) => {
bail!("PipeWire frame channel disconnected");
}
}
}
}
use stats::{pix_fmt, BenchArgs, FrameStats};
fn main() -> Result<()> {
let bench_args = BenchArgs::parse();
@@ -123,7 +64,7 @@ fn main() -> Result<()> {
println!("[1/4] Portal connected, PipeWire stream active\n");
println!("[2/4] Waiting for first frame from PipeWire...");
let first_frame = receive_first_frame(&cap)?;
let first_frame = common::receive_first_frame(&cap)?;
let src_width = first_frame.width;
let src_height = first_frame.height;
@@ -142,6 +83,9 @@ fn main() -> Result<()> {
println!("[3/4] Testing mmap on DMA-BUF...");
let mmap_size = (src_stride as usize) * (src_height as usize);
// SAFETY: first_frame.fd is an open DMA-BUF; offset/size come from PipeWire's
// negotiated format. PROT_READ+MAP_SHARED is the standard read-only DMA-BUF
// mapping. Returns MAP_FAILED on error (checked below).
let mmap_ptr = unsafe {
libc::mmap(
ptr::null_mut(),
@@ -173,6 +117,8 @@ fn main() -> Result<()> {
"[3/4] mmap SUCCESS — CPU can read DMA-BUF ({:.1} MB)\n",
mmap_size as f64 / 1024.0 / 1024.0
);
// SAFETY: mmap_ptr was returned by mmap above and is not MAP_FAILED (checked);
// mmap_size matches the original mapping. POSIX munmap(2) releases the mapping.
unsafe {
libc::munmap(mmap_ptr, mmap_size);
}
@@ -205,6 +151,9 @@ fn main() -> Result<()> {
let codec_name = codec.name();
if codec_name == "libx264" {
// SAFETY: enc is a valid AVCodecContext for the not-yet-opened encoder;
// priv_data is the x264 private options struct. All CStrings live across
// both av_opt_set calls. These set the x264 "preset" and "tune" options.
unsafe {
let key = CString::new("preset").unwrap();
let val = CString::new("veryfast").unwrap();
@@ -220,6 +169,8 @@ fn main() -> Result<()> {
// Create output format context via FFI
let mut fmt_ctx_ptr: *mut ffi::AVFormatContext = ptr::null_mut();
// SAFETY: fmt_ctx_ptr is an out-parameter initialized by FFmpeg; output_cstr
// lives across the call. Returns 0 on success; we check below.
let ret = unsafe {
ffi::avformat_alloc_output_context2(
&mut fmt_ctx_ptr,
@@ -232,21 +183,30 @@ fn main() -> Result<()> {
bail!("Failed to allocate output format context: error {ret}");
}
// SAFETY: fmt_ctx_ptr is the valid output context allocated above.
// avformat_new_stream returns a pointer to a new AVStream or NULL on failure.
let stream_ptr = unsafe { ffi::avformat_new_stream(fmt_ctx_ptr, ptr::null()) };
if stream_ptr.is_null() {
bail!("Failed to create new stream");
}
// SAFETY: stream_ptr and enc_video.as_ptr() are valid pointers; codecpar is
// the output destination inside stream. avcodec_parameters_from_context copies
// encoder parameters into the stream's codecpar.
let ret =
unsafe { ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr()) };
if ret < 0 {
bail!("Failed to copy encoder parameters: error {ret}");
}
// SAFETY: stream_ptr and enc_video are valid; time_base is a plain AVRational
// field copied from encoder to stream.
unsafe {
(*stream_ptr).time_base = (*enc_video.as_ptr()).time_base;
}
// SAFETY: fmt_ctx_ptr is valid; pb is the AVIOContext slot to initialize;
// output_cstr is a valid NUL-terminated path; AVIO_FLAG_WRITE is a constant.
let ret = unsafe {
ffi::avio_open(
&mut (*fmt_ctx_ptr).pb,
@@ -261,17 +221,23 @@ fn main() -> Result<()> {
);
}
// SAFETY: fmt_ctx_ptr is fully configured (streams + pb set); NULL options
// is the default. Returns 0 on success.
let ret = unsafe { ffi::avformat_write_header(fmt_ctx_ptr, ptr::null_mut()) };
if ret < 0 {
bail!("Failed to write header: error {ret}");
}
// SAFETY: fmt_ctx_ptr is a fully initialized output context (header written).
// Output::wrap takes ownership of the pointer into a safe RAII wrapper.
let mut octx = unsafe { ff::format::context::Output::wrap(fmt_ctx_ptr) };
// Create sws_scale context: BGRZ (BGR0) -> YUV420P
let bgr0_fmt = pix_fmt(ff::format::Pixel::BGRZ);
let yuv420p_fmt = pix_fmt(ff::format::Pixel::YUV420P);
// SAFETY: all parameters are valid enum/pixel format values; NULL filters are
// allowed by FFmpeg. sws_getContext returns a heap-allocated SwsContext or NULL.
let sws_ctx = unsafe {
ffi::sws_getContext(
src_width as i32,
@@ -291,6 +257,9 @@ fn main() -> Result<()> {
}
// Allocate reusable YUV frame
// SAFETY: av_frame_alloc returns NULL only on OOM. After allocation we set
// width/height/format fields and call av_frame_get_buffer to allocate plane
// data. On failure we free the frame via av_frame_free before bailing.
let mut yuv_frame = unsafe {
let mut f = ffi::av_frame_alloc();
if f.is_null() {
@@ -349,6 +318,9 @@ fn main() -> Result<()> {
let mmap_start = Instant::now();
let frame_size = (frame.stride as usize) * (frame.height as usize);
// SAFETY: frame.fd is an open DMA-BUF owned by the frame; offset/size come
// from PipeWire's negotiated format. PROT_READ+MAP_SHARED for read-only
// DMA-BUF access. Returns MAP_FAILED on error (checked below).
let mmap_ptr = unsafe {
libc::mmap(
ptr::null_mut(),
@@ -369,8 +341,15 @@ fn main() -> Result<()> {
stats.mmap_us.push(mmap_start.elapsed().as_micros() as u64);
let scale_start = Instant::now();
// SAFETY: mmap_ptr is a valid mapping of frame_size bytes (checked above);
// constructing a read-only slice over it for the duration of sws_scale is
// sound as long as we don't hold it past munmap (we don't).
let src_data = unsafe { std::slice::from_raw_parts(mmap_ptr as *const u8, frame_size) };
// SAFETY: yuv_frame and sws_ctx are valid; src_data is a valid slice of the
// mmap'd DMA-BUF for this frame. sws_scale reads src planes (BGR0 -> YUV420P)
// and writes into yuv_frame's data planes. av_frame_make_writable ensures
// yuv_frame is not shared before writing.
unsafe {
ffi::av_frame_make_writable(yuv_frame);
@@ -391,6 +370,8 @@ fn main() -> Result<()> {
.scale_us
.push(scale_start.elapsed().as_micros() as u64);
// SAFETY: mmap_ptr was returned by mmap above and is not MAP_FAILED; frame_size
// matches the original mapping. Release before dropping frame (which closes fd).
unsafe {
libc::munmap(mmap_ptr, frame_size);
}
@@ -398,6 +379,9 @@ fn main() -> Result<()> {
let encode_start = Instant::now();
// SAFETY: yuv_frame is allocated and writable; enc_video is the opened encoder.
// Setting pts is a plain i64 field write. avcodec_send_frame submits the frame
// for encoding; returns < 0 on error (we log and continue).
unsafe {
(*yuv_frame).pts = pts;
pts += 1;
@@ -409,7 +393,7 @@ fn main() -> Result<()> {
}
}
drain_encoder(&mut enc_video, &mut octx)?;
common::drain_encoder(&mut enc_video, &mut octx)?;
stats
.encode_us
@@ -419,7 +403,7 @@ fn main() -> Result<()> {
.push(frame_start.elapsed().as_micros() as u64);
frames_encoded += 1;
if frames_encoded % 30 == 0 {
if frames_encoded.is_multiple_of(30) {
let fps = frames_encoded as f64 / total_start.elapsed().as_secs_f64();
println!(
" [{}/{}] {:.1} FPS",
@@ -431,15 +415,20 @@ fn main() -> Result<()> {
let total_elapsed = total_start.elapsed();
println!("\nFlushing encoder...");
// SAFETY: enc_video is the opened encoder; passing NULL frame signals EOF to
// drain the encoder's internal pipeline. Returns < 0 on error (ignored here).
unsafe {
ffi::avcodec_send_frame(enc_video.as_mut_ptr(), ptr::null());
}
drain_encoder(&mut enc_video, &mut octx)?;
common::drain_encoder(&mut enc_video, &mut octx)?;
octx.write_trailer()
.map_err(|e| anyhow::anyhow!("Failed to write trailer: {e}"))?;
// Cleanup
// SAFETY: yuv_frame is the allocated frame from earlier (still owned by us);
// sws_ctx is the allocated sws context. av_frame_free and sws_freeContext take
// ownership and free their respective heap allocations.
unsafe {
ffi::av_frame_free(&mut yuv_frame as *mut _);
ffi::sws_freeContext(sws_ctx);
@@ -519,32 +508,3 @@ fn main() -> Result<()> {
println!("Output written to: {}", bench_args.output);
Ok(())
}
fn drain_encoder(
enc_video: &mut ff::encoder::video::Video,
octx: &mut ff::format::context::Output,
) -> Result<()> {
loop {
let mut pkt = ff::Packet::empty();
let ret = unsafe { ffi::avcodec_receive_packet(enc_video.as_mut_ptr(), pkt.as_mut_ptr()) };
if ret < 0 {
if ret == ffi::AVERROR(ffi::EAGAIN) || ret == ffi::AVERROR_EOF {
break;
}
eprintln!("avcodec_receive_packet failed: {ret}");
break;
}
let enc_tb = enc_video.time_base();
let stream_tb = unsafe {
let streams = (*octx.as_ptr()).streams;
let st = *streams.add(0);
ff::Rational::from((*st).time_base)
};
pkt.rescale_ts(enc_tb, stream_tb);
pkt.set_stream(0);
pkt.write_interleaved(octx)
.map_err(|e| anyhow::anyhow!("write packet failed: {e}"))?;
}
Ok(())
}
+45
View File
@@ -0,0 +1,45 @@
use clap::Parser;
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
#[derive(Parser, Debug)]
#[command(
name = "sw_encode_bench",
about = "Software encoding pipeline benchmark"
)]
pub(crate) struct BenchArgs {
#[arg(short, long)]
pub(crate) output: String,
#[arg(long, default_value_t = 120)]
pub(crate) frames: u32,
#[arg(long, default_value_t = 2560)]
pub(crate) enc_width: u32,
#[arg(long, default_value_t = 1440)]
pub(crate) enc_height: u32,
}
#[derive(Default)]
pub(crate) struct FrameStats {
pub(crate) mmap_us: Vec<u64>,
pub(crate) scale_us: Vec<u64>,
pub(crate) encode_us: Vec<u64>,
pub(crate) total_us: Vec<u64>,
pub(crate) mmap_failures: u32,
}
impl FrameStats {
pub(crate) fn avg_ms(data: &[u64]) -> f64 {
if data.is_empty() {
return 0.0;
}
data.iter().sum::<u64>() as f64 / data.len() as f64 / 1000.0
}
}
pub(crate) fn pix_fmt(p: ff::format::Pixel) -> ffi::AVPixelFormat {
Into::<ffi::AVPixelFormat>::into(p)
}
File diff suppressed because it is too large Load Diff
+218
View File
@@ -0,0 +1,218 @@
// vaapi_import_bench.rs — VAAPI DMA-BUF import + GPU-side downscale benchmark
//
// Tests: Portal capture -> av_hwframe_map (ARGB sw_format) -> transfer -> sw encode
//
// Usage: cargo run --bin vaapi_import_bench -- --output /tmp/vaapi_bench.mp4
use std::os::fd::AsRawFd;
use std::path::Path;
use std::ptr;
use std::time::Instant;
use anyhow::Result;
use clap::Parser;
use ffmpeg_next as ff;
use wl_webrtc::args::Args;
use wl_webrtc::avhw::{import_dma_buf_to_vaapi, AvHwDevCtx, AvHwFrameCtx};
use wl_webrtc::cap_portal::CapPortal;
#[path = "../common/mod.rs"]
mod common;
mod pipeline_cpu;
mod pipeline_gpu;
mod software;
mod stats;
mod util;
use pipeline_cpu::run_cpu_pipeline;
use pipeline_gpu::run_gpu_pipeline;
use stats::{BenchArgs, PipelineMode};
use util::{output_for_mode, print_comparison, print_detailed_results};
fn main() -> Result<()> {
let bench_args = BenchArgs::parse();
println!("=== VAAPI Import Benchmark ===");
println!("Output: {}", bench_args.output);
println!("Target frames: {}", bench_args.frames);
println!(
"Encode resolution: {}x{}",
bench_args.enc_width, bench_args.enc_height
);
println!("DRM device: {}", bench_args.drm_device);
println!();
ff::init()?;
println!("[1/3] Requesting screen capture via XDG Portal...");
println!(" (Select a screen to share in the portal dialog)");
let portal_args = Args {
output: Some(bench_args.output.clone()),
output_name: None,
fps: 60,
codec: "h264".to_string(),
hw_accel: "vaapi".to_string(),
drm_device: None,
bitrate: None,
max_bitrate: 8_000_000,
gop_size: None,
verbose: false,
backend: Some("portal".to_string()),
port: 0,
no_persist: false,
stats: false,
};
let cap = CapPortal::new(&portal_args)?;
println!("[1/3] Portal connected, PipeWire stream active\n");
println!("[2/3] Waiting for first frame from PipeWire...");
let first_frame = common::receive_first_frame(&cap)?;
let src_width = first_frame.width;
let src_height = first_frame.height;
let src_format = first_frame.format;
println!(
"[2/3] First frame: {}x{}, format=0x{:08X}, stride={}, modifier=0x{:X}",
src_width, src_height, src_format, first_frame.stride, first_frame.modifier
);
println!("\n[2/3] Testing av_hwframe_map with sw_format=BGRA...");
println!(
" DRM format chain: PipeWire BGRA -> DRM_FORMAT_ARGB8888 (0x{:08X}) -> VA_FOURCC_BGRA -> AV_PIX_FMT_BGRA",
src_format
);
let drm_device = Path::new(&bench_args.drm_device);
let hw_dev = AvHwDevCtx::new_vaapi(drm_device)?;
println!(" VAAPI device context created OK");
let frames_ctx =
AvHwFrameCtx::for_capture(&hw_dev, src_width, src_height, ff::format::Pixel::BGRA)?;
println!(" VAAPI frames context created OK (sw_format=BGRA)");
// SAFETY: delegates to avhw::import_dma_buf_to_vaapi (itself an unsafe fn).
// frames_ctx is a valid AVBufferRef from AvHwFrameCtx::for_capture above;
// `first_frame` is the PipeWire-formatted PwDmaBufFrame whose metadata the
// function reads directly. See that function's own SAFETY contract for the
// full rationale.
let vaapi_frame = unsafe { import_dma_buf_to_vaapi(frames_ctx.as_ptr(), &first_frame) };
match &vaapi_frame {
Ok(_) => {
println!(" Result: SUCCESS — av_hwframe_map imported DMA-BUF to VAAPI surface!");
}
Err(e) => {
println!(" Result: FAILED");
println!(" Error: {e}");
println!();
println!(" Possible causes:");
println!(" - sw_format mismatch (current: BGRA)");
println!(" - DRM format modifier not supported by VAAPI");
println!(" - VAAPI driver doesn't support DMA-BUF import for this format");
println!();
println!(" Falling back to mmap readback test for comparison...");
let mmap_size = (first_frame.stride as usize) * (first_frame.height as usize);
let mmap_start = Instant::now();
// SAFETY: first_frame.fd is an open DMA-BUF; offset/size from PipeWire.
// PROT_READ+MAP_SHARED is the standard read-only DMA-BUF mapping. Returns
// MAP_FAILED on error (checked below).
let mmap_ptr = unsafe {
libc::mmap(
ptr::null_mut(),
mmap_size,
libc::PROT_READ,
libc::MAP_SHARED,
first_frame.fd.as_raw_fd(),
first_frame.offset as i64,
)
};
let mmap_elapsed = mmap_start.elapsed();
if mmap_ptr == libc::MAP_FAILED {
let errno = std::io::Error::last_os_error();
println!(" mmap also FAILED: {errno}");
} else {
println!(
" mmap SUCCESS: {:.1} MB, setup in {:.2}ms",
mmap_size as f64 / 1024.0 / 1024.0,
mmap_elapsed.as_secs_f64() * 1000.0
);
// SAFETY: mmap_ptr is a valid mapping (MAP_FAILED path was handled
// above); mmap_size matches the original mapping. POSIX munmap(2).
unsafe {
libc::munmap(mmap_ptr, mmap_size);
}
}
println!();
println!("=== Benchmark ended: av_hwframe_map import FAILED ===");
println!("Fix the import issue before proceeding to GPU downscale tests.");
return Ok(());
}
}
drop(vaapi_frame);
drop(first_frame);
println!("\n[3/3] Benchmarking selected pipeline(s)...");
let enc_width = bench_args.enc_width;
let enc_height = bench_args.enc_height;
let split_outputs = bench_args.mode == PipelineMode::Both;
let mut cpu_stats = None;
let mut gpu_stats = None;
if matches!(bench_args.mode, PipelineMode::Cpu | PipelineMode::Both) {
let output = output_for_mode(&bench_args.output, PipelineMode::Cpu, split_outputs);
cpu_stats = Some(run_cpu_pipeline(
&cap,
&frames_ctx,
&output,
bench_args.frames,
src_width,
src_height,
enc_width,
enc_height,
)?);
}
if matches!(bench_args.mode, PipelineMode::Gpu | PipelineMode::Both) {
let output = output_for_mode(&bench_args.output, PipelineMode::Gpu, split_outputs);
gpu_stats = Some(run_gpu_pipeline(
&cap,
&hw_dev,
&frames_ctx,
&output,
bench_args.frames,
src_width,
src_height,
enc_width,
enc_height,
)?);
}
if let Some(stats) = cpu_stats.as_ref() {
print_detailed_results("CPU", stats, src_width, src_height, enc_width, enc_height);
}
if let Some(stats) = gpu_stats.as_ref() {
print_detailed_results("GPU", stats, src_width, src_height, enc_width, enc_height);
}
print_comparison(cpu_stats.as_ref(), gpu_stats.as_ref());
if cpu_stats
.as_ref()
.into_iter()
.chain(gpu_stats.as_ref())
.any(|stats| stats.achieved_fps() < 30.0 && stats.frames_encoded > 0)
{
println!("NOTE: At least one achieved FPS result is below 30 FPS target.");
}
Ok(())
}
+152
View File
@@ -0,0 +1,152 @@
use std::path::Path;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next::ffi;
use wl_webrtc::avhw::{av_err_to_string, AvHwFrameCtx};
use wl_webrtc::cap_portal::{CapPortal, PwCtrlEvent};
use crate::pipeline_gpu::import_frame;
use crate::software::{
create_software_encoder, create_sws_context, encode_yuv_frame, finish_encoder,
};
use crate::stats::FrameStats;
#[allow(clippy::too_many_arguments)]
pub(crate) fn run_cpu_pipeline(
cap: &CapPortal,
frames_ctx: &AvHwFrameCtx,
output: &str,
frames: u32,
src_width: u32,
src_height: u32,
enc_width: u32,
enc_height: u32,
) -> Result<FrameStats> {
let mut encoder = create_software_encoder(Path::new(output), enc_width, enc_height)?;
let sws_ctx = create_sws_context(
src_width,
src_height,
ffi::AVPixelFormat::AV_PIX_FMT_BGRA,
enc_width,
enc_height,
)?;
println!(
" Encoder: {}, {}x{} YUV420P",
encoder.codec_name, enc_width, enc_height
);
println!(" Output: {output}");
println!(" CPU Pipeline: DMA-BUF 4K BGRA -> av_hwframe_map -> av_hwframe_transfer_data -> sws_scale -> YUV420P 2K -> encode\n");
let mut stats = FrameStats {
codec_name: encoder.codec_name.clone(),
output_path: output.to_string(),
..FrameStats::default()
};
let total_start = Instant::now();
let mut pts: i64 = 0;
while stats.frames_encoded < frames {
if let Ok(ctrl) = cap.event_receiver().try_recv() {
match ctrl {
PwCtrlEvent::StreamEnded => break,
PwCtrlEvent::Error(e) => bail!(
"PipeWire error after {} CPU frames: {e}",
stats.frames_encoded
),
PwCtrlEvent::FormatChanged { .. } => {}
}
}
let frame = match cap
.frame_receiver()
.recv_timeout(std::time::Duration::from_secs(5))
{
Ok(f) => f,
Err(_) => break,
};
let frame_start = Instant::now();
let t_import = Instant::now();
let vaapi_frame = match import_frame(frames_ctx, &frame) {
Ok(f) => f,
Err(e) => {
stats.import_failures += 1;
if stats.import_failures <= 3 {
eprintln!("CPU frame {}: import failed: {e}", stats.frames_encoded);
}
continue;
}
};
let import_us = t_import.elapsed().as_micros() as u64;
let t_transfer = Instant::now();
// SAFETY: sw_frame is allocated by FFmpeg and freed on all paths below.
let mut sw_frame = unsafe { ffi::av_frame_alloc() };
if sw_frame.is_null() {
bail!("CPU frame {}: av_frame_alloc failed", stats.frames_encoded);
}
// SAFETY: sw_frame is an allocated destination; vaapi_frame is a valid VAAPI source frame.
let transfer_ret =
unsafe { ffi::av_hwframe_transfer_data(sw_frame, vaapi_frame.as_ptr(), 0) };
if transfer_ret < 0 {
// SAFETY: sw_frame was allocated above and has not been freed yet.
unsafe { ffi::av_frame_free(&mut sw_frame) };
bail!(
"CPU frame {}: av_hwframe_transfer_data failed: {} ({})",
stats.frames_encoded,
transfer_ret,
av_err_to_string(transfer_ret)
);
}
let transfer_us = t_transfer.elapsed().as_micros() as u64;
let t_scale = Instant::now();
// SAFETY: sw_frame contains transferred BGRA data; encoder.yuv_frame is writable YUV420P
// at the configured output dimensions; sws_ctx converts and downscales between them.
unsafe {
ffi::av_frame_make_writable(encoder.yuv_frame);
ffi::sws_scale(
sws_ctx.0,
(*sw_frame).data.as_ptr() as *const *const u8,
(*sw_frame).linesize.as_ptr(),
0,
(*sw_frame).height,
(*encoder.yuv_frame).data.as_ptr() as *mut *mut u8,
(*encoder.yuv_frame).linesize.as_ptr(),
);
}
let scale_us = t_scale.elapsed().as_micros() as u64;
// SAFETY: sw_frame was allocated above and is no longer needed after scaling.
unsafe { ffi::av_frame_free(&mut sw_frame) };
let encode_us = encode_yuv_frame(&mut encoder, &mut pts)?;
let total_us = frame_start.elapsed().as_micros() as u64;
stats.import_us.push(import_us);
stats.transfer_us.push(transfer_us);
stats.scale_us.push(scale_us);
stats.encode_us.push(encode_us);
stats.total_us.push(total_us);
stats.frames_encoded += 1;
if stats.frames_encoded <= 3 || stats.frames_encoded.is_multiple_of(30) {
println!(
" CPU frame {:>4}/{frames}: import={:.2}ms transfer={:.2}ms scale={:.2}ms encode={:.2}ms total={:.2}ms",
stats.frames_encoded,
import_us as f64 / 1000.0,
transfer_us as f64 / 1000.0,
scale_us as f64 / 1000.0,
encode_us as f64 / 1000.0,
total_us as f64 / 1000.0,
);
}
}
finish_encoder(encoder)?;
stats.elapsed_secs = total_start.elapsed().as_secs_f64();
Ok(stats)
}
+250
View File
@@ -0,0 +1,250 @@
use std::path::Path;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use wl_webrtc::avhw::{av_err_to_string, import_dma_buf_to_vaapi, AvHwDevCtx, AvHwFrameCtx};
use wl_webrtc::cap_portal::{CapPortal, PwCtrlEvent};
use crate::software::{
create_software_encoder, create_sws_context, encode_yuv_frame, finish_encoder,
};
use crate::stats::FrameStats;
pub(crate) fn import_frame(
frames_ctx: &AvHwFrameCtx,
frame: &wl_webrtc::cap_portal::PwDmaBufFrame,
) -> Result<ff::frame::Video> {
// SAFETY: frames_ctx is a live VAAPI frames context configured for the capture format; frame
// carries a valid DMA-BUF fd and metadata from PipeWire for the duration of the call.
// SAFETY: frames_ctx is a valid VAAPI frames context; `frame` carries the
// DMA-BUF metadata read by the function.
unsafe { import_dma_buf_to_vaapi(frames_ctx.as_ptr(), frame) }
}
fn build_gpu_filter_graph(
hw_dev: &AvHwDevCtx,
frames_rgb: &AvHwFrameCtx,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
) -> Result<ff::filter::Graph> {
let mut graph = ff::filter::Graph::new();
let buffersrc =
ff::filter::find("buffer").ok_or_else(|| anyhow::anyhow!("filter 'buffer' not found"))?;
let buffersink = ff::filter::find("buffersink")
.ok_or_else(|| anyhow::anyhow!("filter 'buffersink' not found"))?;
let scale_vaapi = ff::filter::find("scale_vaapi")
.ok_or_else(|| anyhow::anyhow!("filter 'scale_vaapi' not found"))?;
// pix_fmt must be set via av_buffersrc_parameters_set (below), not in args —
// FFmpeg 8.0+ rejects HW pixel formats during init() if hw_frames_ctx is missing.
// Use a placeholder SW format here; it gets overridden by parameters_set below.
let args = format!(
"video_size={}x{}:pix_fmt=bgra:time_base=1/60:pixel_aspect=1/1",
width, height,
);
let mut src_ctx = graph.add(&buffersrc, "in", &args)?;
// SAFETY: Allocate buffersrc parameters, attach a ref-counted hw_frames_ctx compatible with
// imported VAAPI BGRA frames, apply it, then free only the parameter struct (not the ref).
let par = unsafe { ffi::av_buffersrc_parameters_alloc() };
if par.is_null() {
bail!("av_buffersrc_parameters_alloc returned null");
}
// SAFETY: par and src_ctx are valid; frames_rgb.ref_clone returns an owned AVBufferRef.
unsafe {
(*par).format = Into::<ffi::AVPixelFormat>::into(ff::format::Pixel::VAAPI) as i32;
(*par).width = width as i32;
(*par).height = height as i32;
(*par).time_base = ffi::AVRational { num: 1, den: 60 };
(*par).hw_frames_ctx = frames_rgb.ref_clone();
let ret = ffi::av_buffersrc_parameters_set(src_ctx.as_mut_ptr(), par);
ffi::av_free(par as *mut _);
if ret < 0 {
bail!("av_buffersrc_parameters_set failed: error {ret}");
}
}
let mut scale_ctx = graph.add(
&scale_vaapi,
"scale",
&format!("{enc_width}:{enc_height}:format=nv12"),
)?;
// SAFETY: scale_vaapi uses this ref-counted VAAPI device context while graph is alive.
unsafe {
(*scale_ctx.as_mut_ptr()).hw_device_ctx = hw_dev.ref_clone();
}
let mut sink_ctx = graph.add(&buffersink, "out", "")?;
src_ctx.link(0, &mut scale_ctx, 0);
scale_ctx.link(0, &mut sink_ctx, 0);
graph
.validate()
.map_err(|e| anyhow::anyhow!("GPU filter graph validation failed: {e}"))?;
Ok(graph)
}
#[allow(clippy::too_many_arguments)]
pub(crate) fn run_gpu_pipeline(
cap: &CapPortal,
hw_dev: &AvHwDevCtx,
frames_ctx: &AvHwFrameCtx,
output: &str,
frames: u32,
src_width: u32,
src_height: u32,
enc_width: u32,
enc_height: u32,
) -> Result<FrameStats> {
let mut encoder = create_software_encoder(Path::new(output), enc_width, enc_height)?;
let format_ctx = create_sws_context(
enc_width,
enc_height,
ffi::AVPixelFormat::AV_PIX_FMT_NV12,
enc_width,
enc_height,
)?;
let mut graph = build_gpu_filter_graph(
hw_dev, frames_ctx, src_width, src_height, enc_width, enc_height,
)?;
println!(
" Encoder: {}, {}x{} YUV420P",
encoder.codec_name, enc_width, enc_height
);
println!(" Output: {output}");
println!(" GPU Pipeline: DMA-BUF 4K BGRA -> av_hwframe_map -> scale_vaapi 2K NV12 -> transfer small NV12 -> sws_scale format-only -> encode\n");
let mut stats = FrameStats {
codec_name: encoder.codec_name.clone(),
output_path: output.to_string(),
..FrameStats::default()
};
let total_start = Instant::now();
let mut pts: i64 = 0;
while stats.frames_encoded < frames {
if let Ok(ctrl) = cap.event_receiver().try_recv() {
match ctrl {
PwCtrlEvent::StreamEnded => break,
PwCtrlEvent::Error(e) => bail!(
"PipeWire error after {} GPU frames: {e}",
stats.frames_encoded
),
PwCtrlEvent::FormatChanged { .. } => {}
}
}
let frame = match cap
.frame_receiver()
.recv_timeout(std::time::Duration::from_secs(5))
{
Ok(f) => f,
Err(_) => break,
};
let frame_start = Instant::now();
let t_import = Instant::now();
let vaapi_frame = match import_frame(frames_ctx, &frame) {
Ok(f) => f,
Err(e) => {
stats.import_failures += 1;
if stats.import_failures <= 3 {
eprintln!("GPU frame {}: import failed: {e}", stats.frames_encoded);
}
continue;
}
};
let import_us = t_import.elapsed().as_micros() as u64;
let t_filter = Instant::now();
let mut filter_src_ctx = graph.get("in").unwrap();
let mut filter_src = filter_src_ctx.source();
let mut filter_sink_ctx = graph.get("out").unwrap();
let mut filter_sink = filter_sink_ctx.sink();
filter_src
.add(&vaapi_frame)
.map_err(|e| anyhow::anyhow!("GPU filter source add failed: {e}"))?;
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => {}
Err(ff::Error::Other { errno }) if errno == ffi::EAGAIN => continue,
Err(e) => bail!("GPU filter sink get frame failed: {e}"),
}
let filter_us = t_filter.elapsed().as_micros() as u64;
let t_transfer = Instant::now();
// SAFETY: sw_nv12 is allocated by FFmpeg and freed after format conversion.
let mut sw_nv12 = unsafe { ffi::av_frame_alloc() };
if sw_nv12.is_null() {
bail!("GPU frame {}: av_frame_alloc failed", stats.frames_encoded);
}
// SAFETY: sw_nv12 is an allocated destination; filtered is a valid 2K NV12 VAAPI frame.
let transfer_ret = unsafe { ffi::av_hwframe_transfer_data(sw_nv12, filtered.as_ptr(), 0) };
if transfer_ret < 0 {
// SAFETY: sw_nv12 was allocated above and has not been freed yet.
unsafe { ffi::av_frame_free(&mut sw_nv12) };
bail!(
"GPU frame {}: av_hwframe_transfer_data failed: {} ({})",
stats.frames_encoded,
transfer_ret,
av_err_to_string(transfer_ret)
);
}
let transfer_us = t_transfer.elapsed().as_micros() as u64;
let t_format = Instant::now();
// SAFETY: sw_nv12 contains CPU-side NV12 at enc dimensions; encoder.yuv_frame is writable
// YUV420P at the same dimensions, so sws_scale performs only chroma deinterleave/format conversion.
unsafe {
ffi::av_frame_make_writable(encoder.yuv_frame);
ffi::sws_scale(
format_ctx.0,
(*sw_nv12).data.as_ptr() as *const *const u8,
(*sw_nv12).linesize.as_ptr(),
0,
(*sw_nv12).height,
(*encoder.yuv_frame).data.as_ptr() as *mut *mut u8,
(*encoder.yuv_frame).linesize.as_ptr(),
);
}
let format_us = t_format.elapsed().as_micros() as u64;
// SAFETY: sw_nv12 was allocated above and is no longer needed.
unsafe { ffi::av_frame_free(&mut sw_nv12) };
let encode_us = encode_yuv_frame(&mut encoder, &mut pts)?;
let total_us = frame_start.elapsed().as_micros() as u64;
stats.import_us.push(import_us);
stats.filter_us.push(filter_us);
stats.transfer_us.push(transfer_us);
stats.format_us.push(format_us);
stats.encode_us.push(encode_us);
stats.total_us.push(total_us);
stats.frames_encoded += 1;
if stats.frames_encoded <= 3 || stats.frames_encoded.is_multiple_of(30) {
println!(
" GPU frame {:>4}/{frames}: import={:.2}ms filter={:.2}ms transfer={:.2}ms format={:.2}ms encode={:.2}ms total={:.2}ms",
stats.frames_encoded,
import_us as f64 / 1000.0,
filter_us as f64 / 1000.0,
transfer_us as f64 / 1000.0,
format_us as f64 / 1000.0,
encode_us as f64 / 1000.0,
total_us as f64 / 1000.0,
);
}
}
finish_encoder(encoder)?;
stats.elapsed_secs = total_start.elapsed().as_secs_f64();
Ok(stats)
}
+228
View File
@@ -0,0 +1,228 @@
use std::ffi::CString;
use std::path::Path;
use std::ptr;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use crate::common::drain_encoder;
pub(crate) struct SoftwareEncoder {
pub(crate) enc_video: ff::codec::encoder::video::Video,
pub(crate) octx: ff::format::context::Output,
pub(crate) yuv_frame: *mut ffi::AVFrame,
pub(crate) codec_name: String,
}
impl Drop for SoftwareEncoder {
fn drop(&mut self) {
// SAFETY: yuv_frame is allocated by av_frame_alloc in create_software_encoder and
// owned exclusively by this SoftwareEncoder.
unsafe {
ffi::av_frame_free(&mut self.yuv_frame);
}
}
}
pub(crate) struct SwsContext(pub(crate) *mut ffi::SwsContext);
impl Drop for SwsContext {
fn drop(&mut self) {
// SAFETY: Context is either null or returned by sws_getContext and owned here.
unsafe {
ffi::sws_freeContext(self.0);
}
}
}
pub(crate) fn create_software_encoder(
output_path: &Path,
width: u32,
height: u32,
) -> Result<SoftwareEncoder> {
let output_cstr = CString::new(output_path.to_str().unwrap())?;
let codec = ff::encoder::find_by_name("libx264")
.or_else(|| ff::encoder::find_by_name("libopenh264"))
.ok_or_else(|| {
anyhow::anyhow!("No H.264 software encoder found (tried libx264, libopenh264)")
})?;
let codec_name = codec.name().to_string();
let mut enc = {
let ctx = ff::codec::Context::new_with_codec(codec);
ctx.encoder().video()?
};
enc.set_width(width);
enc.set_height(height);
enc.set_format(ff::format::Pixel::YUV420P);
enc.set_time_base(ff::Rational::new(1, 60));
enc.set_max_b_frames(0);
enc.set_gop(60);
if codec_name == "libx264" {
// SAFETY: priv_data belongs to the not-yet-opened encoder context. Option strings are
// valid NUL-terminated C strings for the duration of each av_opt_set call.
unsafe {
let key = CString::new("preset").unwrap();
let val = CString::new("veryfast").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("tune").unwrap();
let val = CString::new("zerolatency").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
}
}
let opened = enc.open()?;
let enc_video = opened.0;
let use_null_muxer = output_path
.to_str()
.map(|s| s.contains("null"))
.unwrap_or(false);
let fmt_name = if use_null_muxer {
CString::new("null").unwrap()
} else {
CString::new("").unwrap()
};
let fmt_name_ptr = if use_null_muxer {
fmt_name.as_ptr()
} else {
ptr::null()
};
let mut fmt_ctx_ptr: *mut ffi::AVFormatContext = ptr::null_mut();
// SAFETY: fmt_ctx_ptr is an out parameter initialized by FFmpeg; output_cstr and fmt_name live
// across the call.
let ret = unsafe {
ffi::avformat_alloc_output_context2(
&mut fmt_ctx_ptr,
ptr::null_mut(),
fmt_name_ptr,
output_cstr.as_ptr(),
)
};
if ret < 0 || fmt_ctx_ptr.is_null() {
bail!("Failed to allocate output format context: error {ret}");
}
// SAFETY: fmt_ctx_ptr is a valid output context allocated above.
let stream_ptr = unsafe { ffi::avformat_new_stream(fmt_ctx_ptr, ptr::null()) };
if stream_ptr.is_null() {
bail!("Failed to create output stream");
}
// SAFETY: stream and codec context pointers are valid; parameters are copied into stream.
let ret =
unsafe { ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr()) };
if ret < 0 {
bail!("Failed to copy codec parameters: error {ret}");
}
// SAFETY: fmt_ctx_ptr is valid; pb is initialized for non-NOFILE muxers.
unsafe {
if (*(*fmt_ctx_ptr).oformat).flags & ffi::AVFMT_NOFILE == 0 {
let ret = ffi::avio_open(
&mut (*fmt_ctx_ptr).pb,
output_cstr.as_ptr(),
ffi::AVIO_FLAG_WRITE,
);
if ret < 0 {
bail!("Failed to open output file: error {ret}");
}
}
}
// SAFETY: fmt_ctx_ptr is a fully configured output context.
let ret = unsafe { ffi::avformat_write_header(fmt_ctx_ptr, ptr::null_mut()) };
if ret < 0 {
bail!("Failed to write header: error {ret}");
}
// SAFETY: ownership of fmt_ctx_ptr transfers into ffmpeg-next Output wrapper.
let octx = unsafe { ff::format::context::Output::wrap(fmt_ctx_ptr) };
// SAFETY: Allocate and configure an owned writable YUV420P frame for encoder input.
let yuv_frame = unsafe {
let mut f = ffi::av_frame_alloc();
if f.is_null() {
bail!("av_frame_alloc failed");
}
(*f).width = width as i32;
(*f).height = height as i32;
(*f).format = ffi::AVPixelFormat::AV_PIX_FMT_YUV420P as i32;
let r = ffi::av_frame_get_buffer(f, 0);
if r < 0 {
ffi::av_frame_free(&mut f);
bail!("av_frame_get_buffer failed: {r}");
}
f
};
Ok(SoftwareEncoder {
enc_video,
octx,
yuv_frame,
codec_name,
})
}
pub(crate) fn create_sws_context(
src_width: u32,
src_height: u32,
src_fmt: ffi::AVPixelFormat,
dst_width: u32,
dst_height: u32,
) -> Result<SwsContext> {
// SAFETY: sws_getContext creates an owned scaler context for the provided dimensions/formats.
let ctx = unsafe {
ffi::sws_getContext(
src_width as i32,
src_height as i32,
src_fmt,
dst_width as i32,
dst_height as i32,
ffi::AVPixelFormat::AV_PIX_FMT_YUV420P,
2,
ptr::null_mut(),
ptr::null_mut(),
ptr::null_mut(),
)
};
if ctx.is_null() {
bail!("Failed to create sws_scale context");
}
Ok(SwsContext(ctx))
}
pub(crate) fn encode_yuv_frame(encoder: &mut SoftwareEncoder, pts: &mut i64) -> Result<u64> {
let t_encode = Instant::now();
// SAFETY: yuv_frame is allocated, writable, and formatted as the encoder's configured
// YUV420P input frame. FFmpeg consumes but does not take ownership.
unsafe {
(*encoder.yuv_frame).pts = *pts;
*pts += 1;
let r = ffi::avcodec_send_frame(encoder.enc_video.as_mut_ptr(), encoder.yuv_frame);
if r < 0 {
bail!("avcodec_send_frame failed: {r}");
}
}
drain_encoder(&mut encoder.enc_video, &mut encoder.octx)?;
Ok(t_encode.elapsed().as_micros() as u64)
}
pub(crate) fn finish_encoder(mut encoder: SoftwareEncoder) -> Result<()> {
// SAFETY: Sending a null frame flushes the encoder; context remains owned by encoder.
unsafe {
ffi::avcodec_send_frame(encoder.enc_video.as_mut_ptr(), ptr::null());
}
drain_encoder(&mut encoder.enc_video, &mut encoder.octx)?;
encoder
.octx
.write_trailer()
.map_err(|e| anyhow::anyhow!("Failed to write trailer: {e}"))?;
Ok(())
}
+76
View File
@@ -0,0 +1,76 @@
use clap::{Parser, ValueEnum};
#[derive(Parser, Debug)]
#[command(name = "vaapi_import_bench", about = "VAAPI DMA-BUF import benchmark")]
pub(crate) struct BenchArgs {
#[arg(short, long)]
pub(crate) output: String,
#[arg(long, default_value_t = 60)]
pub(crate) frames: u32,
#[arg(long, default_value_t = 2560)]
pub(crate) enc_width: u32,
#[arg(long, default_value_t = 1440)]
pub(crate) enc_height: u32,
#[arg(long, default_value = "/dev/dri/renderD128")]
pub(crate) drm_device: String,
#[arg(long, value_enum, default_value_t = PipelineMode::Both)]
pub(crate) mode: PipelineMode,
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, ValueEnum)]
pub(crate) enum PipelineMode {
Cpu,
Gpu,
Both,
}
#[derive(Default)]
pub(crate) struct FrameStats {
pub(crate) import_us: Vec<u64>,
pub(crate) filter_us: Vec<u64>,
pub(crate) transfer_us: Vec<u64>,
pub(crate) scale_us: Vec<u64>,
pub(crate) format_us: Vec<u64>,
pub(crate) encode_us: Vec<u64>,
pub(crate) total_us: Vec<u64>,
pub(crate) import_failures: u32,
pub(crate) frames_encoded: u32,
pub(crate) elapsed_secs: f64,
pub(crate) codec_name: String,
pub(crate) output_path: String,
}
impl FrameStats {
pub(crate) fn avg_ms(data: &[u64]) -> f64 {
if data.is_empty() {
return 0.0;
}
data.iter().sum::<u64>() as f64 / data.len() as f64 / 1000.0
}
pub(crate) fn avg_total_ms(&self) -> f64 {
Self::avg_ms(&self.total_us)
}
pub(crate) fn achieved_fps(&self) -> f64 {
if self.frames_encoded > 0 && self.elapsed_secs > 0.0 {
self.frames_encoded as f64 / self.elapsed_secs
} else {
0.0
}
}
pub(crate) fn theoretical_fps(&self) -> f64 {
let avg = self.avg_total_ms();
if avg > 0.0 {
1000.0 / avg
} else {
0.0
}
}
}
+107
View File
@@ -0,0 +1,107 @@
use std::path::Path;
use crate::stats::{FrameStats, PipelineMode};
pub(crate) fn output_for_mode(base: &str, mode: PipelineMode, split: bool) -> String {
if !split || base.contains("null") {
return base.to_string();
}
let path = Path::new(base);
let suffix = match mode {
PipelineMode::Cpu => "cpu",
PipelineMode::Gpu => "gpu",
PipelineMode::Both => unreachable!(),
};
let file_name = path.file_name().and_then(|s| s.to_str()).unwrap_or(base);
let split_name = if let Some((stem, ext)) = file_name.rsplit_once('.') {
format!("{stem}.{suffix}.{ext}")
} else {
format!("{file_name}.{suffix}")
};
path.with_file_name(split_name)
.to_string_lossy()
.into_owned()
}
pub(crate) fn print_detailed_results(
label: &str,
stats: &FrameStats,
src_width: u32,
src_height: u32,
enc_width: u32,
enc_height: u32,
) {
println!();
println!("=== {label} Pipeline Results ===");
println!("Capture resolution: {}x{}", src_width, src_height);
println!("Encode resolution: {}x{}", enc_width, enc_height);
println!("Frames encoded: {}", stats.frames_encoded);
println!("Total time: {:.2}s", stats.elapsed_secs);
println!("Output: {}", stats.output_path);
if stats.import_failures > 0 {
println!("Import failures: {}", stats.import_failures);
}
println!(
"import avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.import_us)
);
if !stats.filter_us.is_empty() {
println!(
"filter avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.filter_us)
);
}
println!(
"transfer avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.transfer_us)
);
if !stats.scale_us.is_empty() {
println!(
"scale avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.scale_us)
);
}
if !stats.format_us.is_empty() {
println!(
"format avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.format_us)
);
}
println!(
"encode ({}): {:.2} ms/frame",
stats.codec_name,
FrameStats::avg_ms(&stats.encode_us)
);
println!("total avg: {:.2} ms/frame", stats.avg_total_ms());
println!("achieved FPS: {:.1}", stats.achieved_fps());
println!("max theoretical: {:.1} FPS", stats.theoretical_fps());
}
pub(crate) fn print_comparison(cpu: Option<&FrameStats>, gpu: Option<&FrameStats>) {
println!();
println!("=== Pipeline Comparison ===");
if let Some(s) = cpu {
println!(
"CPU: import={:.2}ms transfer={:.2}ms scale={:.2}ms encode={:.2}ms total={:.2}ms ({:.1} FPS)",
FrameStats::avg_ms(&s.import_us),
FrameStats::avg_ms(&s.transfer_us),
FrameStats::avg_ms(&s.scale_us),
FrameStats::avg_ms(&s.encode_us),
s.avg_total_ms(),
s.theoretical_fps(),
);
}
if let Some(s) = gpu {
println!(
"GPU: import={:.2}ms filter={:.2}ms transfer={:.2}ms format={:.2}ms encode={:.2}ms total={:.2}ms ({:.1} FPS)",
FrameStats::avg_ms(&s.import_us),
FrameStats::avg_ms(&s.filter_us),
FrameStats::avg_ms(&s.transfer_us),
FrameStats::avg_ms(&s.format_us),
FrameStats::avg_ms(&s.encode_us),
s.avg_total_ms(),
s.theoretical_fps(),
);
}
}
+39 -1086
View File
File diff suppressed because it is too large Load Diff
+73
View File
@@ -0,0 +1,73 @@
/// 将 PipeWire SPA 视频格式转换为 DRM FourCC 格式
///
/// PipeWire 使用自己的 VideoFormat 枚举,而 DRM/KMS 使用 FourCC 格式标识。
/// 此函数建立了两者之间的映射关系。
///
/// 支持的格式:
/// 不支持的格式返回 0
/// DRM 格式名描述像素值位布局(大端序),而非内存字节序。
/// 例如 DRM_FORMAT_ARGB8888 在小端 x86 上内存为 [B,G,R,A] = PipeWire BGRA。
pub(super) fn spa_to_drm_fourcc(format: libspa::param::video::VideoFormat) -> u32 {
use drm_fourcc::DrmFourcc;
use libspa::param::video::VideoFormat;
match format {
VideoFormat::BGRA => DrmFourcc::Argb8888 as u32,
VideoFormat::BGRx => DrmFourcc::Xrgb8888 as u32,
VideoFormat::RGBA => DrmFourcc::Abgr8888 as u32,
VideoFormat::RGBx => DrmFourcc::Xbgr8888 as u32,
VideoFormat::ARGB => DrmFourcc::Bgra8888 as u32,
VideoFormat::xRGB => DrmFourcc::Bgrx8888 as u32,
VideoFormat::ABGR => DrmFourcc::Rgba8888 as u32,
VideoFormat::xBGR => DrmFourcc::Rgbx8888 as u32,
_ => 0,
}
}
#[cfg(test)]
mod tests {
use super::*;
use drm_fourcc::DrmFourcc;
#[test]
fn spa_to_drm_fourcc_all_32bit() {
use libspa::param::video::VideoFormat;
assert_eq!(
spa_to_drm_fourcc(VideoFormat::BGRA),
DrmFourcc::Argb8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::BGRx),
DrmFourcc::Xrgb8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::RGBA),
DrmFourcc::Abgr8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::RGBx),
DrmFourcc::Xbgr8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::ARGB),
DrmFourcc::Bgra8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::xRGB),
DrmFourcc::Bgrx8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::ABGR),
DrmFourcc::Rgba8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::xBGR),
DrmFourcc::Rgbx8888 as u32
);
}
#[test]
fn spa_to_drm_fourcc_unsupported() {
use libspa::param::video::VideoFormat;
assert_eq!(spa_to_drm_fourcc(VideoFormat::NV12), 0);
}
}
+18
View File
@@ -0,0 +1,18 @@
/// Log an actionable diagnostic when a Portal phase times out.
///
/// Mirrors the message format from `backend_detect.rs::log_portal_unresponsive`
/// but additionally suggests `--no-persist` when the timeout occurred in a
/// phase that was using a restore token.
pub(super) fn log_portal_phase_timeout(phase: &str, used_restore_token: bool) {
let persist_hint = if used_restore_token {
" If this recurs, try: wl-webrtc --no-persist"
} else {
""
};
tracing::error!(
"Portal service did not respond within timeout while {phase}. \
This usually means xdg-desktop-portal or xdg-desktop-portal-kde is stuck. \
Try: systemctl --user restart xdg-desktop-portal xdg-desktop-portal-kde, \
then re-run wl-webrtc.{persist_hint}"
);
}
+446
View File
@@ -0,0 +1,446 @@
use std::os::fd::{AsRawFd, FromRawFd, OwnedFd};
use std::sync::atomic::AtomicU64;
use std::sync::atomic::Ordering;
use std::sync::Arc;
use std::thread::{self, JoinHandle};
use anyhow::Result;
use crossbeam_channel::Sender;
use super::fourcc::spa_to_drm_fourcc;
use super::types::{PortalFormatInfo, PwCtrlEvent, PwDmaBufFrame};
/// PipeWire 捕获线程的上下文数据
///
/// 从主线程传递给 PipeWire 捕获线程的所有必要资源。
/// 该结构体在线程创建时一次性 move 到线程中使用。
struct PwThreadCtx {
frame_tx: Sender<PwDmaBufFrame>,
event_tx: Sender<PwCtrlEvent>,
dropped: Arc<AtomicU64>,
shutdown_read: OwnedFd,
pw_fd: OwnedFd,
node_id: u32,
}
fn pipewire_thread(ctx: PwThreadCtx) {
use pipewire as pw;
use pw::properties::properties;
use pw::spa::param::video::VideoInfoRaw;
use pw::stream::{StreamBox, StreamFlags};
use std::cell::Cell;
use std::rc::Rc;
// 初始化 PipeWire 进程全局库。
//
// pipewire-rs 内部使用 OnceCell 保护 pw::init(),确保只调用一次。
// pw::deinit() 是 unsafe 且要求"进程生命周期内仅调用一次,且所有
// PipeWire 使用已停止"。由于 CapPortal 可被多次创建销毁,此函数
// 不调用 pw::deinit()——进程退出时全局状态由 OS 回收。
pw::init();
let PwThreadCtx {
frame_tx,
event_tx,
dropped,
shutdown_read,
pw_fd,
node_id,
} = ctx;
let mainloop = match pw::main_loop::MainLoopBox::new(None) {
Ok(ml) => ml,
Err(e) => {
if let Err(e) =
event_tx.try_send(PwCtrlEvent::Error(format!("MainLoop::new failed: {e}")))
{
tracing::error!("MainLoop::new failed and error channel also failed: {e}");
}
return;
}
};
let context = match pw::context::ContextBox::new(mainloop.loop_(), None) {
Ok(c) => c,
Err(e) => {
if let Err(e) =
event_tx.try_send(PwCtrlEvent::Error(format!("Context::new failed: {e}")))
{
tracing::error!("Context::new failed and error channel also failed: {e}");
}
return;
}
};
let core = match context.connect_fd(pw_fd, None) {
Ok(c) => c,
Err(e) => {
if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("connect_fd failed: {e}")))
{
tracing::error!("connect_fd failed and error channel also failed: {e}");
}
return;
}
};
// 创建 PipeWire 视频流
// 属性配置:
// - MEDIA_TYPE = "Video": 媒体类型为视频
// - MEDIA_CATEGORY = "Capture": 类别为捕获(而非回放)
// - MEDIA_ROLE = "Screen": 角色为屏幕(用于策略管理)
let stream = match StreamBox::new(
&core,
"wl-webrtc",
properties! {
*pw::keys::MEDIA_TYPE => "Video",
*pw::keys::MEDIA_CATEGORY => "Capture",
*pw::keys::MEDIA_ROLE => "Screen",
*pw::keys::NODE_FORCE_QUANTUM => "512",
},
) {
Ok(s) => s,
Err(e) => {
if let Err(e) =
event_tx.try_send(PwCtrlEvent::Error(format!("Stream::new failed: {e}")))
{
tracing::error!("Stream::new failed and error channel also failed: {e}");
}
return;
}
};
let format_info: Rc<Cell<Option<PortalFormatInfo>>> = Rc::new(Cell::new(None));
let event_tx_state = event_tx.clone();
let _listener = stream
.add_local_listener::<()>()
.state_changed(move |_, _, old, new| {
tracing::info!("PipeWire stream state: {old:?} -> {new:?}");
match new {
pw::stream::StreamState::Error(e) => {
tracing::error!("PipeWire stream error: {e}");
let _ = event_tx_state.try_send(PwCtrlEvent::StreamEnded);
}
pw::stream::StreamState::Unconnected => {
let _ = event_tx_state.try_send(PwCtrlEvent::StreamEnded);
}
pw::stream::StreamState::Paused => {
tracing::warn!("PipeWire stream paused (compositor may be switching content)");
}
pw::stream::StreamState::Streaming => {
tracing::info!("PipeWire stream (re)started");
}
pw::stream::StreamState::Connecting => {}
}
})
// 参数变化回调(格式协商)
// PipeWire 在流格式协商完成后触发此回调
// id 为参数类型,param 包含具体的格式参数(分辨率、像素格式等)
.param_changed({
let format_info = format_info.clone();
let event_tx = event_tx.clone();
move |_, _, id, param| {
// 仅处理 Format 类型的参数变化
let Some(param) = param else { return };
if id != pw::spa::param::ParamType::Format.as_raw() {
return;
}
// 解析视频格式信息(分辨率、像素格式、修饰符等)
let mut info = VideoInfoRaw::new();
if let Err(e) = info.parse(param) {
tracing::warn!("Failed to parse video format: {e}");
return;
}
let width = info.size().width;
let height = info.size().height;
// 将 SPA 视频格式转换为 DRM FourCC 格式标识符
let drm_format = spa_to_drm_fourcc(info.format());
// 获取 DRM 修饰符,描述 GPU buffer 的内存布局(如 tiling 模式)
let modifier = info.modifier();
let framerate = info.framerate();
let max_framerate = info.max_framerate();
// 保存协商后的格式信息,供 process 回调读取
let previous_format = format_info.get();
format_info.set(Some(PortalFormatInfo {
width,
height,
drm_format,
modifier,
}));
if let Some(prev) = previous_format {
if width != prev.width || height != prev.height {
tracing::warn!(
"PipeWire dimensions changed: {}x{} (format renegotiation)",
width,
height
);
let _ = event_tx.try_send(PwCtrlEvent::FormatChanged { width, height });
}
}
tracing::info!(
"PipeWire format negotiated: {width}x{height}, \
drm_format={drm_format:#010x}, modifier={modifier:#x}, \
framerate={}/{}, max_framerate={}/{}",
framerate.num,
framerate.denom,
max_framerate.num,
max_framerate.denom,
);
}
})
// 帧处理回调 —— 这是核心的数据路径
// 每当 PipeWire 有新的帧数据可用时触发
// 关键操作: 从 buffer 中提取 DMA-BUF fddup 后通过 channel 发送给消费者
.process({
let format_info = format_info.clone();
let frame_tx = frame_tx.clone();
move |stream, _| {
// SAFETY: raw_buf ownership invariant — PipeWire's process callback
// contract requires that every buffer acquired via `dequeue_raw_buffer`
// is returned to the queue EXACTLY ONCE via `queue_raw_buffer` before
// the callback returns — on every exit path, success or error. Failure
// to requeue leaks the buffer slot and eventually stalls the stream.
//
// Audit map of this closure (verified 2026-06-28):
// - null raw_buf (dequeue returned NULL) → nothing to requeue, return.
// - null spa_buf / no data / bad fd / null chunk / no format_info /
// invalid dims / dup_fd < 0 → all requeue before early-return.
// - success (try_send Ok / Full / Disconnected) → final requeue at end.
// The fd ownership is independent: dup() creates a fresh fd that lives
// inside PwDmaBufFrame; on try_send error the frame Drops and closes it.
let raw_buf = unsafe { stream.dequeue_raw_buffer() };
if raw_buf.is_null() {
tracing::trace!("process: null raw_buf");
return;
}
// 获取 SPA buffer 结构体,包含数据数组、元数据等
// SAFETY: raw_buf was checked non-null above. `pw_buffer.buffer` is a
// valid raw pointer for the lifetime of raw_buf (PipeWire keeps the
// buffer alive until we queue it back).
let spa_buf = unsafe { (*raw_buf).buffer };
if spa_buf.is_null() {
tracing::trace!("process: null spa_buf");
// SAFETY: raw_buf is the non-null buffer we still own; returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
// 获取 buffer 中的数据项数量和数据指针
// 对于 DMA-BUF 帧,通常只有 1 个数据项(包含 fd)
// SAFETY: spa_buf checked non-null above; `n_datas` is a plain u32 field.
let n_datas = unsafe { (*spa_buf).n_datas };
// SAFETY: same as above; `datas` is a raw pointer field, may be null.
let datas_ptr = unsafe { (*spa_buf).datas };
if n_datas == 0 || datas_ptr.is_null() {
tracing::trace!("process: no data (n_datas={n_datas})");
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
// 从第一个数据项中获取 DMA-BUF 文件描述符
// 通过 libspa 的 Data 包装类型安全地访问 SPA 数据结构
// SAFETY: datas_ptr is non-null and n_datas > 0 (checked above). We cast
// to pw::spa::buffer::Data and take a shared borrow; PipeWire does not
// mutate the data array during a process cycle, so a shared reference
// for the duration of this callback is sound.
let data_ref: &pw::spa::buffer::Data =
unsafe { &*(datas_ptr as *const pw::spa::buffer::Data) };
let fd = data_ref.fd();
if fd < 0 {
tracing::trace!("process: invalid fd={fd}");
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
if data_ref.as_raw().chunk.is_null() {
tracing::trace!("process: null chunk");
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
let chunk = data_ref.chunk();
let offset = chunk.offset() as u64;
let stride = chunk.stride() as u32;
// 从 SPA_META_Header 元数据中提取 PTS (显示时间戳)
// 遍历 buffer 的所有元数据项,查找 Header 类型的元数据
// PTS 可用于音视频同步和帧率控制
// SAFETY: spa_buf is non-null. `metas` is checked for null before
// iteration. We iterate `i in 0..n_metas` reading shared POD fields
// (type_, size, data) — PipeWire keeps the meta array immutable during
// a process cycle. The size guard (`meta.size >= size_of::<spa_meta_header>()`)
// and null-data check before reading ensure we never read past the
// meta's actual extent.
let pts: i64 = unsafe {
let mut pts_val: i64 = 0;
let n_metas = (*spa_buf).n_metas;
let metas = (*spa_buf).metas;
if !metas.is_null() {
for i in 0..n_metas {
let meta = &*metas.add(i as usize);
if meta.type_ == libspa::sys::SPA_META_Header
&& meta.size as usize
>= std::mem::size_of::<libspa::sys::spa_meta_header>()
&& !meta.data.is_null()
{
let header = &*(meta.data as *const libspa::sys::spa_meta_header);
pts_val = header.pts;
break;
}
}
}
pts_val
};
// 验证格式信息已协商完成,且分辨率和格式有效
let Some(fmt) = format_info.get() else {
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
};
let PortalFormatInfo {
width,
height,
drm_format: format,
modifier,
} = fmt;
if width == 0 || height == 0 || format == 0 {
tracing::trace!("process: invalid dimensions {width}x{height} format={format}");
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
// 复制 DMA-BUF 文件描述符
// 必须 dup,因为原始 fd 由 PipeWire 管理,我们不能持有它
// dup 后的 fd 由 PwDmaBufFrame 持有,生命周期独立于 PipeWire buffer
// SAFETY: `fd` is the open DMA-BUF fd reported by PipeWire (>= 0 checked
// above). libc::dup is the standard POSIX fd duplication call. The
// original `fd` remains owned by PipeWire (returned with raw_buf later).
let dup_fd = unsafe { libc::dup(fd) };
if dup_fd < 0 {
// SAFETY: raw_buf still owned, returning it. No fd cleanup needed
// because dup() failed and never returned a new fd.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
// 构建帧数据对象,所有必要的帧信息已收集完毕
// SAFETY: `dup_fd` is a freshly-dup'd open file descriptor (>= 0 checked
// above) and we are its sole owner. OwnedFd::from_raw_fd takes ownership
// and will close() it on Drop. The fd's lifecycle is independent of
// raw_buf: whether try_send succeeds (frame moves into the channel) or
// fails (Full/Disconnected — the error payload owns the frame and drops
// it at the end of the match arm), exactly one close() occurs per dup().
let frame_fd = unsafe { OwnedFd::from_raw_fd(dup_fd) };
let frame = PwDmaBufFrame {
fd: frame_fd,
offset,
stride,
modifier,
width,
height,
format,
pts,
};
match frame_tx.try_send(frame) {
Ok(()) => {}
Err(crossbeam_channel::TrySendError::Full(_)) => {
dropped.fetch_add(1, Ordering::Relaxed);
}
Err(crossbeam_channel::TrySendError::Disconnected(_)) => {}
}
// SAFETY: final exactly-once requeue of raw_buf. Every path above
// either returned early with its own requeue, or falls through to here.
unsafe { stream.queue_raw_buffer(raw_buf) };
}
})
.register();
let mut params: [&pw::spa::pod::Pod; 0] = [];
if let Err(e) = stream.connect(
pw::spa::utils::Direction::Input,
Some(node_id),
StreamFlags::AUTOCONNECT | StreamFlags::MAP_BUFFERS,
&mut params,
) {
if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("stream.connect failed: {e}")))
{
tracing::error!("stream.connect failed and error channel also failed: {e}");
}
return;
}
let loop_ = mainloop.loop_();
// Register the shutdown eventfd on the PipeWire loop.
//
// When CapPortal::drop writes to the eventfd, the loop wakes up and
// dispatches this callback on the loop thread. Because the callback
// only fires while mainloop.run() is blocking this thread, mainloop
// is guaranteed alive — eliminating the UAF that existed with the
// previous detached helper thread approach.
// 保存 mainloop 的原始指针,用于在 shutdown 回调中调用 pw_main_loop_quit
// 这是安全的,因为回调只在 mainloop.run() 阻塞期间执行
let mainloop_ptr = mainloop.as_raw_ptr();
let _shutdown_source = loop_.add_io(
shutdown_read,
libspa::support::system::IoFlags::IN,
move |fd| {
// Drain the eventfd so it doesn't re-trigger
let mut buf: u64 = 0;
// SAFETY: `fd` is the registered eventfd owned by the mainloop source; the
// buffer is a stack u64 of 8 bytes matching the count argument. POSIX
// read(2) is the standard fd-read syscall; eventfd semantics require the
// 8-byte buffer.
let _ = unsafe {
libc::read(
fd.as_raw_fd(),
&mut buf as *mut u64 as *mut _,
std::mem::size_of::<u64>(),
)
};
// SAFETY: This callback only executes while mainloop.run() is
// blocking this thread, so mainloop is guaranteed alive.
unsafe { pipewire::sys::pw_main_loop_quit(mainloop_ptr) };
},
);
// 启动 PipeWire 主事件循环
// 此调用会阻塞当前线程,直到 mainloop.quit() 被调用
// quit() 由 shutdown eventfd 的 IO 回调触发
mainloop.run();
// run() returned — _shutdown_source drops first (reverse declaration order),
// which unregisters the callback from the loop. Then mainloop drops.
// No dangling raw pointers are possible.
// PipeWire global state is intentionally not deinitialized here — see pw::init() comment above.
}
pub(super) fn spawn_pipewire_thread(
frame_tx: Sender<PwDmaBufFrame>,
event_tx: Sender<PwCtrlEvent>,
dropped: Arc<AtomicU64>,
shutdown_read: OwnedFd,
pw_fd: OwnedFd,
node_id: u32,
) -> Result<JoinHandle<()>> {
let ctx = PwThreadCtx {
frame_tx,
event_tx,
dropped,
shutdown_read,
pw_fd,
node_id,
};
thread::Builder::new()
.name("pipewire-capture".into())
.spawn(move || pipewire_thread(ctx))
.map_err(|e| anyhow::anyhow!("thread spawn failed: {e}"))
}
+192
View File
@@ -0,0 +1,192 @@
use std::os::fd::OwnedFd;
use anyhow::Result;
use super::logging::log_portal_phase_timeout;
use super::token_fs::{delete_restore_token, load_restore_token, save_restore_token};
use super::types::{PortalPhaseTimeout, PORTAL_SERVICE_TIMEOUT, PORTAL_USER_DIALOG_TIMEOUT};
use super::CapPortal;
impl CapPortal {
/// 通过 XDG Desktop Portal 建立屏幕录制会话
///
/// 与桌面环境的 D-Bus 服务交互,请求用户授权屏幕录制。
/// 流程:
/// 1. 创建 Screencast 代理(D-Bus 代理)
/// 2. 创建 ScreenCast 会话
/// 3. 配置源选择参数(光标模式、显示器源、不持久化会话)
/// 4. 启动录制,获取流信息(包含 PipeWire node_id
/// 5. 打开 PipeWire 远程连接,获取文件描述符
///
/// 返回 (PipeWire fd, node_id),供 PipeWire 线程连接使用
///
/// Wraps `_setup_portal_inner` with token-aware retry: on a `TokenDependent`
/// timeout (phases 3 or 4 with a restore token in use) AND `no_persist ==
/// false`, clears the cached restore token and retries once with
/// `no_persist = true`.
pub(super) async fn setup_portal(no_persist: bool) -> Result<(OwnedFd, u32)> {
match Self::_setup_portal_inner(no_persist, false).await {
Ok(result) => Ok(result),
Err(e) if e.is::<PortalPhaseTimeout>() => {
let inner_err = e.downcast_ref::<PortalPhaseTimeout>().unwrap();
match inner_err {
PortalPhaseTimeout::TokenDependent if !no_persist => {
tracing::warn!(
"Portal timed out during token-using phase. \
Clearing cached restore token and retrying with fresh authorization."
);
delete_restore_token();
Self::_setup_portal_inner(true, true).await
}
_ => Err(e),
}
}
Err(e) => Err(e),
}
}
/// Inner Portal setup with phased timeouts. See `setup_portal` for the
/// retry wrapper.
///
/// `is_retry == true` disables further retry attempts (max 1 retry).
pub(super) async fn _setup_portal_inner(
no_persist: bool,
is_retry: bool,
) -> Result<(OwnedFd, u32)> {
use ashpd::desktop::screencast::{
CursorMode, Screencast, SelectSourcesOptions, SourceType,
};
use ashpd::desktop::PersistMode;
// Phase 1: Screencast proxy (no user interaction).
let proxy = match tokio::time::timeout(PORTAL_SERVICE_TIMEOUT, Screencast::new()).await {
Ok(Ok(p)) => p,
Ok(Err(e)) => return Err(anyhow::anyhow!("Failed to create Screencast proxy: {e}")),
Err(_) => {
log_portal_phase_timeout("creating Screencast proxy", false);
return Err(PortalPhaseTimeout::Service.into());
}
};
// Phase 2: create_session (no user interaction).
let session = match tokio::time::timeout(
PORTAL_SERVICE_TIMEOUT,
proxy.create_session(Default::default()),
)
.await
{
Ok(Ok(s)) => s,
Ok(Err(e)) => return Err(anyhow::anyhow!("Failed to create ScreenCast session: {e}")),
Err(_) => {
log_portal_phase_timeout("creating session", false);
return Err(PortalPhaseTimeout::Service.into());
}
};
let version_supported = proxy.version() >= 4;
let (persist_mode, saved_token) = if !no_persist && version_supported {
let token = load_restore_token();
if token.is_some() {
if is_retry {
tracing::info!("Re-attempting portal session after token clear");
} else {
tracing::info!("Attempting to restore portal session with saved token");
}
}
(PersistMode::ExplicitlyRevoked, token)
} else {
(PersistMode::DoNot, None)
};
let mut options = SelectSourcesOptions::default()
.set_cursor_mode(CursorMode::Embedded)
.set_sources(ashpd::enumflags2::BitFlags::from(SourceType::Monitor))
.set_multiple(false)
.set_persist_mode(persist_mode);
if let Some(ref token) = saved_token {
options = options.set_restore_token(token.as_str());
}
// Phase 3: select_sources — token path is fast (no dialog); fresh
// authorization may pop a dialog.
let token_in_use = saved_token.is_some();
let phase3_timeout = if token_in_use {
PORTAL_SERVICE_TIMEOUT
} else {
PORTAL_USER_DIALOG_TIMEOUT
};
match tokio::time::timeout(phase3_timeout, proxy.select_sources(&session, options)).await {
Ok(Ok(_)) => {}
Ok(Err(e)) => return Err(anyhow::anyhow!("Screen sharing permission denied: {e}")),
Err(_) => {
log_portal_phase_timeout("selecting sources", token_in_use);
return Err(if token_in_use {
PortalPhaseTimeout::TokenDependent
} else {
PortalPhaseTimeout::Service
}
.into());
}
}
// Phase 4: start + response — same dialog-vs-token reasoning as phase 3.
let phase4_timeout = if token_in_use {
PORTAL_SERVICE_TIMEOUT
} else {
PORTAL_USER_DIALOG_TIMEOUT
};
let start_fut = async {
proxy
.start(&session, None, Default::default())
.await?
.response()
};
let response = match tokio::time::timeout(phase4_timeout, start_fut).await {
Ok(Ok(r)) => r,
Ok(Err(e)) => return Err(anyhow::anyhow!("ScreenCast start/response error: {e}")),
Err(_) => {
log_portal_phase_timeout("starting session", token_in_use);
return Err(if token_in_use {
PortalPhaseTimeout::TokenDependent
} else {
PortalPhaseTimeout::Service
}
.into());
}
};
if !no_persist && version_supported {
if let Some(new_token) = response.restore_token() {
save_restore_token(new_token);
}
}
let stream = response
.streams()
.first()
.ok_or_else(|| anyhow::anyhow!("No streams returned from ScreenCast"))?;
let node_id = stream.pipe_wire_node_id();
// Phase 5: open_pipe_wire_remote (no user interaction).
let fd = match tokio::time::timeout(
PORTAL_SERVICE_TIMEOUT,
proxy.open_pipe_wire_remote(&session, Default::default()),
)
.await
{
Ok(Ok(f)) => f,
Ok(Err(e)) => return Err(anyhow::anyhow!("Failed to open PipeWire remote: {e}")),
Err(_) => {
log_portal_phase_timeout("opening PipeWire remote", false);
return Err(PortalPhaseTimeout::Service.into());
}
};
tracing::info!("Portal session established: node_id={node_id}");
Ok((fd, node_id))
}
}
+362
View File
@@ -0,0 +1,362 @@
use std::path::PathBuf;
pub(super) fn token_path() -> Option<PathBuf> {
dirs::cache_dir().map(|base| base.join("wl-webrtc").join("portal-restore-token"))
}
/// Verify that `path` is a directory owned by the current user with no group/other permissions.
/// Rejects symlinks at the path itself (but allows the resolved target to be a real dir).
pub(super) fn verify_secure_dir(path: &std::path::Path) -> bool {
use std::os::unix::fs::{MetadataExt, PermissionsExt};
match std::fs::symlink_metadata(path) {
Ok(meta) => {
if meta.file_type().is_symlink() {
tracing::warn!(
"Token parent dir is a symlink, rejecting: {}",
path.display()
);
return false;
}
// Must be a directory
if !meta.is_dir() {
tracing::warn!("Token parent path is not a directory: {}", path.display());
return false;
}
// Must be owned by current user
// SAFETY: libc::getuid has no preconditions and cannot fail; it simply
// returns the calling process's real user ID.
// SAFETY: libc::getuid has no preconditions and cannot fail.
if meta.uid() != unsafe { libc::getuid() } {
tracing::warn!(
"Token parent dir not owned by current user: {}",
path.display()
);
return false;
}
// No group or other permissions (mode must be 0o700 exactly within the 0o777 mask)
let mode = meta.permissions().mode() & 0o777;
if mode != 0o700 {
tracing::warn!(
"Token parent dir has insecure permissions {:o}, expected 0700: {}",
mode,
path.display()
);
return false;
}
true
}
Err(e) => {
tracing::warn!("Failed to stat token parent dir: {e}");
false
}
}
}
/// Ensure the parent directory exists with restrictive permissions (0o700).
/// Returns false if the directory could not be created or is insecure.
pub(super) fn ensure_secure_parent(parent: &std::path::Path) -> bool {
use std::os::unix::fs::{DirBuilderExt, PermissionsExt};
if parent.exists() {
// Directory exists — try to tighten permissions, then verify.
// set_permissions follows symlinks, which is fine here since
// we verify with symlink_metadata in verify_secure_dir.
if let Err(e) = std::fs::set_permissions(parent, std::fs::Permissions::from_mode(0o700)) {
tracing::warn!("Failed to set directory permissions: {e}");
return false;
}
return verify_secure_dir(parent);
}
// Create with restrictive mode — DirBuilderExt::mode bypasses umask.
let mut builder = std::fs::DirBuilder::new();
builder.recursive(true);
builder.mode(0o700);
if let Err(e) = builder.create(parent) {
tracing::warn!("Failed to create token directory: {e}");
return false;
}
// Verify after creation (belt-and-suspenders)
verify_secure_dir(parent)
}
pub(super) fn load_restore_token() -> Option<String> {
load_restore_token_from(token_path()?)
}
pub(super) fn load_restore_token_from(path: PathBuf) -> Option<String> {
use std::os::unix::fs::{MetadataExt, PermissionsExt};
let meta = match std::fs::symlink_metadata(&path) {
Ok(m) => m,
Err(_) => return None,
};
if meta.file_type().is_symlink() {
tracing::warn!(
"Token file is a symlink, refusing to read: {}",
path.display()
);
return None;
}
if !meta.is_file() {
tracing::warn!("Token path is not a regular file: {}", path.display());
return None;
}
// SAFETY: libc::getuid has no preconditions and cannot fail.
if meta.uid() != unsafe { libc::getuid() } {
tracing::warn!("Token file not owned by current user: {}", path.display());
return None;
}
let mode = meta.permissions().mode() & 0o777;
if mode & 0o077 != 0 {
tracing::warn!(
"Token file has insecure permissions {:o}, refusing to read: {}",
mode,
path.display()
);
return None;
}
let token = std::fs::read_to_string(&path).ok()?;
let trimmed = token.trim().to_string();
if trimmed.is_empty() {
None
} else {
Some(trimmed)
}
}
pub(super) fn save_restore_token(token: &str) {
let Some(path) = token_path() else {
tracing::warn!("No secure cache directory available, skipping token save");
return;
};
save_restore_token_to(token, &path);
}
pub(super) fn delete_restore_token() {
let Some(path) = token_path() else {
return;
};
match std::fs::remove_file(&path) {
Ok(()) => tracing::info!("Deleted stale portal restore token at {}", path.display()),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => tracing::warn!(
"Failed to delete stale restore token at {}: {e}",
path.display()
),
}
}
pub(super) fn save_restore_token_to(token: &str, path: &std::path::Path) {
use std::fs::OpenOptions;
use std::io::Write;
use std::os::unix::fs::OpenOptionsExt;
let Some(parent) = path.parent() else {
tracing::warn!("Token path has no parent directory");
return;
};
if !ensure_secure_parent(parent) {
tracing::warn!("Parent directory is insecure, refusing to save token");
return;
}
// Use a unique temp file to prevent symlink attacks.
// create_new(true) guarantees exclusive creation — fails if file already exists,
// and does NOT follow existing symlinks.
let tmp_path = path.with_extension(format!("{}.tmp", std::process::id()));
let result = (|| -> std::io::Result<()> {
let mut f = OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o600)
.open(&tmp_path)?;
f.write_all(token.as_bytes())?;
f.sync_all()?;
std::fs::rename(&tmp_path, path)?;
Ok(())
})();
match result {
Ok(()) => tracing::info!("Saved portal restore token"),
Err(e) => {
let _ = std::fs::remove_file(&tmp_path);
tracing::warn!("Failed to save restore token: {e}");
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::os::unix::fs::PermissionsExt;
#[test]
fn token_path_never_uses_tmp() {
assert!(token_path().is_some(), "token_path should resolve on Linux");
let path = token_path().unwrap();
assert!(!path.starts_with("/tmp"), "must not fallback to /tmp");
}
#[test]
fn verify_secure_dir_rejects_wrong_permissions() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path();
// 0o700 should pass
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o700)).unwrap();
assert!(verify_secure_dir(path));
// 0o755 should fail
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o755)).unwrap();
assert!(!verify_secure_dir(path));
// 0o777 should fail
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o777)).unwrap();
assert!(!verify_secure_dir(path));
}
#[test]
fn verify_secure_dir_rejects_non_directory() {
let dir = tempfile::tempdir().unwrap();
let file_path = dir.path().join("not-a-dir");
std::fs::write(&file_path, b"test").unwrap();
assert!(!verify_secure_dir(&file_path));
}
#[test]
fn ensure_secure_parent_creates_with_0700() {
let base = tempfile::tempdir().unwrap();
let new_dir = base.path().join("wl-test-new-dir");
assert!(!new_dir.exists());
assert!(ensure_secure_parent(&new_dir));
assert!(new_dir.is_dir());
let meta = std::fs::symlink_metadata(&new_dir).unwrap();
let mode = meta.permissions().mode() & 0o777;
assert_eq!(
mode, 0o700,
"created directory should be 0700, got {mode:o}"
);
}
#[test]
fn ensure_secure_parent_tightens_existing_dir() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path();
// Simulate an existing directory with loose permissions
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o755)).unwrap();
assert!(ensure_secure_parent(path));
let meta = std::fs::symlink_metadata(path).unwrap();
let mode = meta.permissions().mode() & 0o777;
assert_eq!(
mode, 0o700,
"tightened directory should be 0700, got {mode:o}"
);
}
#[test]
fn save_creates_file_with_0600() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
save_restore_token_to("secret-token-123", &token_path);
assert!(token_path.exists());
let meta = std::fs::symlink_metadata(&token_path).unwrap();
let mode = meta.permissions().mode() & 0o777;
assert_eq!(mode, 0o600, "token file should be 0600, got {mode:o}");
assert_eq!(
std::fs::read_to_string(&token_path).unwrap(),
"secret-token-123"
);
}
#[test]
fn load_reads_secure_file() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
// Write a valid 0o600 token file
use std::os::unix::fs::OpenOptionsExt;
let mut f = std::fs::OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o600)
.open(&token_path)
.unwrap();
std::io::Write::write_all(&mut f, b"my-secret\n").unwrap();
let result = load_restore_token_from(token_path);
assert_eq!(result, Some("my-secret".to_string()));
}
#[test]
fn load_rejects_group_readable_file() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
// Write with 0o640 (group readable) — should be rejected
use std::os::unix::fs::OpenOptionsExt;
let mut f = std::fs::OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o640)
.open(&token_path)
.unwrap();
std::io::Write::write_all(&mut f, b"leaked-token\n").unwrap();
let result = load_restore_token_from(token_path);
assert!(result.is_none(), "should reject group-readable token file");
}
#[test]
fn load_rejects_world_readable_file() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
use std::os::unix::fs::OpenOptionsExt;
let mut f = std::fs::OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o604)
.open(&token_path)
.unwrap();
std::io::Write::write_all(&mut f, b"leaked-token\n").unwrap();
let result = load_restore_token_from(token_path);
assert!(result.is_none(), "should reject world-readable token file");
}
#[test]
fn load_rejects_symlink() {
let dir = tempfile::tempdir().unwrap();
let real_path = dir.path().join("real-file");
let link_path = dir.path().join("portal-restore-token");
std::fs::write(&real_path, b"target-content\n").unwrap();
std::os::unix::fs::symlink(&real_path, &link_path).unwrap();
let result = load_restore_token_from(link_path);
assert!(result.is_none(), "should reject symlinked token file");
}
#[test]
fn save_then_load_roundtrip() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
save_restore_token_to("roundtrip-token", &token_path);
let loaded = load_restore_token_from(token_path);
assert_eq!(loaded, Some("roundtrip-token".to_string()));
}
}
+79
View File
@@ -0,0 +1,79 @@
use std::os::fd::OwnedFd;
pub(super) const PORTAL_SERVICE_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(5);
pub(super) const PORTAL_USER_DIALOG_TIMEOUT: std::time::Duration =
std::time::Duration::from_secs(30);
/// Classification of Portal phase timeouts to drive retry behavior.
#[derive(Debug)]
pub(super) enum PortalPhaseTimeout {
/// Portal service unresponsive; not retried (user should restart service).
Service,
/// Timed out in token-dependent phase; retried once after clearing token.
TokenDependent,
}
impl std::fmt::Display for PortalPhaseTimeout {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Service => write!(f, "Portal phase timed out (service)"),
Self::TokenDependent => write!(f, "Portal phase timed out (token-dependent)"),
}
}
}
impl std::error::Error for PortalPhaseTimeout {}
/// PipeWire DMA-BUF 帧数据
///
/// 表示从 PipeWire 流中接收到的一帧视频数据。
/// 帧的像素数据存储在 DMA-BUFLinux 的零拷贝 buffer 共享机制)中,
/// 通过文件描述符 (fd) 引用,消费者通过 mmap 或 DRM 导入来访问像素数据。
pub struct PwDmaBufFrame {
/// DMA-BUF 文件描述符,指向 GPU 显存中的帧缓冲区
pub fd: OwnedFd,
/// 帧数据在 DMA-BUF 中的字节偏移量
pub offset: u64,
/// 每行像素的字节跨度(可能大于 width * bpp,因为可能有对齐填充)
pub stride: u32,
/// DRM 格式修饰符,描述 buffer 的内存布局(如线性布局、tiling 等)
pub modifier: u64,
/// 帧宽度(像素)
pub width: u32,
/// 帧高度(像素)
pub height: u32,
/// DRM FourCC 格式标识符(如 BGRA、RGBA 等)
pub format: u32,
/// 显示时间戳 (PTS, Presentation Time Stamp),单位为纳秒
pub pts: i64,
}
/// PipeWire-negotiated video format snapshot, stashed in a `Cell` for cross-callback
/// sharing (format-change callback writes it; process callback reads it). The four
/// fields are the minimal subset of `PwDmaBufFrame`'s metadata that the process
/// callback needs to construct the frame once a buffer arrives.
///
/// `Copy` is required because we store it inside `Cell<Option<PortalFormatInfo>>`;
/// `Cell` requires its contents to be `Copy` (no borrowed interior state).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct PortalFormatInfo {
pub width: u32,
pub height: u32,
/// DRM FourCC format code (e.g. `0x34325258` for XR24 / XRGB8888).
pub drm_format: u32,
/// DRM format modifier describing buffer layout (linear, tiling, etc.).
pub modifier: u64,
}
/// PipeWire 控制事件枚举
///
/// 从 PipeWire 捕获线程发送给消费者的控制事件。
/// 与帧数据分离,通过独立的 channel 传输,确保控制事件不被帧数据淹没。
pub enum PwCtrlEvent {
/// 流已结束(PipeWire 流断开连接或进入错误状态)
StreamEnded,
/// Format/dimensions changed mid-stream
FormatChanged { width: u32, height: u32 },
/// 发生错误,包含错误描述信息
Error(String),
}
+3 -11
View File
@@ -21,9 +21,9 @@ pub struct CapWlrScreencopy {
}
impl CaptureSource for CapWlrScreencopy {
/// Unit type: wlr-screencopy is fully asynchronous — `alloc_frame()`
/// always returns `None`. The frame object is created by Dispatch
/// impls calling `manager.capture_output()`, not by this method.
/// Unit type: wlr-screencopy is fully asynchronous — frame allocation is
/// driven by Dispatch impls calling `manager.capture_output()`, so there
/// is no synchronous `alloc_frame`-style API on this trait.
type Frame = ();
fn new(
@@ -40,14 +40,6 @@ impl CaptureSource for CapWlrScreencopy {
})
}
fn alloc_frame(&mut self) -> Option<Self::Frame> {
// wlr-screencopy is asynchronous: the Dispatch impl creates a new
// ZwlrScreencopyFrameV1 which triggers the buffer allocation flow
// (buffer event → negotiate format → create DMA-BUF). This method
// always returns None.
None
}
fn queue_copy(&mut self, buffer: &WlBuffer, _qh: &QueueHandle<State<Self>>) {
if let Some(frame) = &self.current_frame {
frame.copy(buffer);
+5 -2
View File
@@ -148,6 +148,9 @@ fn run_wlr_screencopy(args: Args) -> Result<()> {
revents: 0,
};
// timeout=0 表示非阻塞,立即返回当前 fd 状态
// SAFETY: `pfd` is a stack-allocated libc::pollfd initialized above with a
// valid wayland_fd and POLLIN events; nfds=1 matches the single-element
// array; timeout=0 is non-blocking. POSIX poll(2) writes revents in place.
let ret = unsafe { libc::poll(&mut pfd, 1, 0) };
tracing::info!(
"Raw poll on wayland fd={wayland_fd}: ret={ret}, revents={}",
@@ -173,7 +176,7 @@ fn run_wlr_screencopy(args: Args) -> Result<()> {
// 注册 SIGINT / SIGTERM 信号用于优雅退出
// signal_hook_mio 将 Unix 信号转换为 fd 可读事件,
// 这样信号也可以通过 epoll 统一监听,不需要单独的信号处理器
let mut signals = signal_hook_mio::v1_0::Signals::new(&[
let mut signals = signal_hook_mio::v1_0::Signals::new([
signal_hook::consts::SIGINT, // Ctrl+C
signal_hook::consts::SIGTERM, // kill 命令默认信号
])?;
@@ -310,7 +313,7 @@ fn run_portal_pipewire(args: Args) -> Result<()> {
// Set up signal handling only (no Wayland fd needed)
// Portal 后端不需要监听 Wayland fd,只需处理 Unix 信号
// 因为帧数据是通过 PipeWire 独立投递的,不走 Wayland 协议
let mut signals = signal_hook_mio::v1_0::Signals::new(&[
let mut signals = signal_hook_mio::v1_0::Signals::new([
signal_hook::consts::SIGINT,
signal_hook::consts::SIGTERM,
])?;
+19
View File
@@ -0,0 +1,19 @@
use wayland_client::protocol::wl_buffer::WlBuffer;
use wayland_client::{Dispatch, Proxy, QueueHandle};
use crate::state::{CaptureSource, State};
impl<S: CaptureSource> Dispatch<WlBuffer, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &WlBuffer,
event: <WlBuffer as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
if let wayland_client::protocol::wl_buffer::Event::Release = event {
tracing::trace!("WlBuffer released");
}
}
}
+115
View File
@@ -0,0 +1,115 @@
use std::mem;
use std::path::PathBuf;
use wayland_client::{Dispatch, Proxy, QueueHandle};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_buffer_params_v1::{
Event as BufferParamsEvent, ZwpLinuxBufferParamsV1,
};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_feedback_v1::{
Event as DmabufFeedbackEvent, ZwpLinuxDmabufFeedbackV1,
};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_v1::{
Event as DmabufEvent, ZwpLinuxDmabufV1,
};
use crate::state::{CaptureSource, EncConstructionStage, InFlightSurface, State};
impl<S: CaptureSource> Dispatch<ZwpLinuxDmabufV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwpLinuxDmabufV1,
event: <ZwpLinuxDmabufV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
DmabufEvent::Format { .. } => {}
DmabufEvent::Modifier { .. } => {}
_ => {}
}
}
}
impl<S: CaptureSource> Dispatch<ZwpLinuxDmabufFeedbackV1, ()> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZwpLinuxDmabufFeedbackV1,
event: <ZwpLinuxDmabufFeedbackV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
DmabufFeedbackEvent::MainDevice { device } => {
if device.len() >= 8 {
let dev_bytes: [u8; 8] = device[..8].try_into().unwrap_or([0u8; 8]);
let dev = u64::from_ne_bytes(dev_bytes);
let minor = ((dev & 0xFF) | ((dev >> 12) & 0xFFFFFF00)) as u32;
let path = PathBuf::from(format!("/dev/dri/renderD{}", minor));
if path.exists() {
tracing::info!(
"Compositor DRM device: {} (dev_t: {})",
path.display(),
dev
);
state.drm_device_from_compositor = Some(path);
} else {
tracing::warn!(
"Compositor reported DRM device {} (dev_t: {}) but path does not exist",
path.display(),
dev
);
}
} else {
tracing::warn!(
"main_device event with unexpected data length: {}",
device.len()
);
}
}
DmabufFeedbackEvent::FormatTable { .. } => {}
DmabufFeedbackEvent::Done => {}
DmabufFeedbackEvent::TrancheDone => {}
DmabufFeedbackEvent::TrancheTargetDevice { .. } => {}
DmabufFeedbackEvent::TrancheFormats { .. } => {}
DmabufFeedbackEvent::TrancheFlags { .. } => {}
_ => {}
}
}
}
impl<S: CaptureSource> Dispatch<ZwpLinuxBufferParamsV1, ()> for State<S> {
fn event(
state: &mut Self,
proxy: &ZwpLinuxBufferParamsV1,
event: <ZwpLinuxBufferParamsV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
BufferParamsEvent::Created { .. } => {
tracing::debug!("DMA-BUF buffer created");
}
BufferParamsEvent::Failed => {
tracing::error!("DMA-BUF buffer creation failed");
let taken = mem::replace(&mut state.in_flight_surface, InFlightSurface::None);
match taken {
InFlightSurface::CopyQueued { buffer, frame, .. } => {
drop(buffer);
if let EncConstructionStage::Streaming { cap, .. } = &mut state.stage {
cap.on_done_with_frame(frame);
}
}
other => {
state.in_flight_surface = other;
}
}
proxy.destroy();
state.errored = true;
}
_ => {}
}
}
}
+6
View File
@@ -0,0 +1,6 @@
mod buffer;
mod dmabuf;
mod output_mgr;
mod registry;
mod screencopy;
mod wl_output;
+109
View File
@@ -0,0 +1,109 @@
use wayland_client::{event_created_child, Dispatch, Proxy, QueueHandle};
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_manager_v1::ZxdgOutputManagerV1;
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_head_v1::{
self, Event as WlrHeadEvent, ZwlrOutputHeadV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_manager_v1::{
self, Event as WlrOutputManagerEvent, ZwlrOutputManagerV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_mode_v1::ZwlrOutputModeV1;
use crate::state::{CaptureSource, EncConstructionStage, State, WlrHeadInfo};
impl<S: CaptureSource> Dispatch<ZxdgOutputManagerV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZxdgOutputManagerV1,
_event: <ZxdgOutputManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
impl<S: CaptureSource> Dispatch<ZwlrOutputManagerV1, ()> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZwlrOutputManagerV1,
event: <ZwlrOutputManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
WlrOutputManagerEvent::Head { head } => {
let _head: ZwlrOutputHeadV1 = head;
tracing::debug!("wlr output head advertised");
}
WlrOutputManagerEvent::Done { .. } => {
if let EncConstructionStage::ProbingOutputs {
wlr_manager_done,
outputs,
..
} = &mut state.stage
{
*wlr_manager_done = true;
let count = outputs.len();
for idx in 0..count {
state.try_finalize_output(idx);
}
}
}
WlrOutputManagerEvent::Finished => {
tracing::warn!("zwlr_output_manager_v1::Finished received during probing");
}
_ => {}
}
}
event_created_child!(State<S>, ZwlrOutputManagerV1, [
zwlr_output_manager_v1::EVT_HEAD_OPCODE => (ZwlrOutputHeadV1, ()),
]);
}
impl<S: CaptureSource> Dispatch<ZwlrOutputHeadV1, ()> for State<S> {
fn event(
state: &mut Self,
proxy: &ZwlrOutputHeadV1,
event: <ZwlrOutputHeadV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
WlrHeadEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs {
wlr_heads,
wlr_head_proxy_to_name,
..
} = &mut state.stage
{
wlr_heads.entry(name.clone()).or_insert(WlrHeadInfo {});
wlr_head_proxy_to_name.insert(proxy.id(), name);
}
}
WlrHeadEvent::Position { .. } => {}
WlrHeadEvent::Finished => {
tracing::debug!("zwlr_output_head_v1::Finished received");
}
_ => {}
}
}
event_created_child!(State<S>, ZwlrOutputHeadV1, [
zwlr_output_head_v1::EVT_MODE_OPCODE => (ZwlrOutputModeV1, ()),
]);
}
impl<S: CaptureSource> Dispatch<ZwlrOutputModeV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwlrOutputModeV1,
_event: <ZwlrOutputModeV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
+116
View File
@@ -0,0 +1,116 @@
use wayland_client::globals::GlobalListContents;
use wayland_client::protocol::wl_output::WlOutput;
use wayland_client::protocol::wl_registry::WlRegistry;
use wayland_client::{Dispatch, QueueHandle};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_v1::ZwpLinuxDmabufV1;
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_manager_v1::ZxdgOutputManagerV1;
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_manager_v1::ZwlrOutputManagerV1;
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_manager_v1::ZwlrScreencopyManagerV1;
use crate::state::{CaptureSource, EncConstructionStage, OutputId, PartialOutputInfo, State};
impl<S: CaptureSource> Dispatch<WlRegistry, GlobalListContents> for State<S> {
fn event(
state: &mut Self,
registry: &WlRegistry,
event: wayland_client::protocol::wl_registry::Event,
_data: &GlobalListContents,
_conn: &wayland_client::Connection,
qhandle: &QueueHandle<State<S>>,
) {
use wayland_client::protocol::wl_registry::Event as RegistryEvent;
match event {
RegistryEvent::Global {
name,
interface,
version,
} => match interface.as_str() {
"zwlr_screencopy_manager_v1" => {
let v = version.min(3);
tracing::debug!("Binding zwlr_screencopy_manager_v1 v{v} (name={name})");
let mgr: ZwlrScreencopyManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
screencopy_manager, ..
} = &mut state.stage
{
*screencopy_manager = Some(mgr);
}
}
"zwp_linux_dmabuf_v1" => {
let v = version.min(4);
tracing::debug!("Binding zwp_linux_dmabuf_v1 v{v} (name={name})");
let proxy: ZwpLinuxDmabufV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
dmabuf,
dmabuf_feedback,
..
} = &mut state.stage
{
*dmabuf = Some(proxy.clone());
if v >= 4 {
let feedback = proxy.get_default_feedback(qhandle, ());
*dmabuf_feedback = Some(feedback);
}
}
}
"wl_output" => {
let v = version.min(4);
tracing::debug!("Binding wl_output v{v} (name={name})");
let output: WlOutput = registry.bind(name, v, qhandle, OutputId(name));
if let EncConstructionStage::ProbingOutputs {
outputs,
bound_outputs,
output_names,
xdg_output_manager,
..
} = &mut state.stage
{
outputs.push(PartialOutputInfo::default());
bound_outputs.push(output.clone());
output_names.push(name);
if let Some(xdg_mgr) = xdg_output_manager {
let output_id = OutputId(name);
xdg_mgr.get_xdg_output(&output, qhandle, output_id);
}
}
}
"zxdg_output_manager_v1" => {
let v = version.min(3);
tracing::debug!("Binding zxdg_output_manager_v1 v{v} (name={name})");
let xdg_mgr: ZxdgOutputManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
bound_outputs,
xdg_output_manager,
output_names,
..
} = &mut state.stage
{
for (i, output) in bound_outputs.iter().enumerate() {
let oname = output_names.get(i).copied().unwrap_or(0);
let output_id = OutputId(oname);
xdg_mgr.get_xdg_output(output, qhandle, output_id);
}
*xdg_output_manager = Some(xdg_mgr);
}
}
"zwlr_output_manager_v1" => {
let v = version.min(4);
tracing::debug!("Binding zwlr_output_manager_v1 v{v} (name={name})");
let mgr: ZwlrOutputManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
wlr_output_manager, ..
} = &mut state.stage
{
*wlr_output_manager = Some(mgr);
}
}
_ => {}
},
RegistryEvent::GlobalRemove { name } => {
tracing::debug!("Global removed: name={name}");
}
_ => {}
}
}
}
+93
View File
@@ -0,0 +1,93 @@
use wayland_client::{Dispatch, Proxy, QueueHandle};
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_frame_v1::{
Event as ScreencopyFrameEvent, ZwlrScreencopyFrameV1,
};
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_manager_v1::ZwlrScreencopyManagerV1;
use crate::cap_wlr_screencopy::CapWlrScreencopy;
use crate::state::{EncConstructionStage, InFlightSurface, State};
impl Dispatch<ZwlrScreencopyFrameV1, ()> for State<CapWlrScreencopy> {
fn event(
state: &mut Self,
proxy: &ZwlrScreencopyFrameV1,
event: <ZwlrScreencopyFrameV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<CapWlrScreencopy>>,
) {
match event {
// SHM buffer offer — in v3 the compositor enumerates supported buffer
// types (buffer and/or linux_dmabuf) before buffer_done. We only
// support DMA-BUF, so just log and wait for linux_dmabuf / buffer_done.
ScreencopyFrameEvent::Buffer { .. } => {
tracing::debug!("Received SHM Buffer offer — only DMA-BUF capture is supported");
}
ScreencopyFrameEvent::LinuxDmabuf {
format,
width,
height,
} => {
tracing::debug!("Screencopy LinuxDmabuf: format={format}, {width}x{height}");
if !matches!(state.in_flight_surface, InFlightSurface::AllocQueued) {
tracing::warn!("Received LinuxDmabuf while no frame allocation was queued");
return;
}
if matches!(state.stage, EncConstructionStage::EverythingButFmt { .. }) {
state.negotiate_format(format, width, height);
if state.errored {
return;
}
}
if let EncConstructionStage::Streaming { cap, .. } = &mut state.stage {
cap.current_frame = Some(proxy.clone());
}
state.on_frame_allocd((), format, width, height);
}
// v3 terminal event: all buffer offers have been enumerated.
// If still AllocQueued, the compositor never sent linux_dmabuf —
// DMA-BUF screencopy is unsupported, so we must error out.
ScreencopyFrameEvent::BufferDone => {
if matches!(state.in_flight_surface, InFlightSurface::AllocQueued) {
tracing::error!(
"Compositor did not offer DMA-BUF screencopy (only SHM); \
DMA-BUF capture is required"
);
state.in_flight_surface = InFlightSurface::None;
proxy.destroy();
state.errored = true;
}
}
ScreencopyFrameEvent::Ready {
tv_sec_hi,
tv_sec_lo,
tv_nsec,
} => {
let tv_sec = (tv_sec_hi as u64) << 32 | tv_sec_lo as u64;
let tv_usec = tv_nsec / 1000;
tracing::trace!("Screencopy ready: tv_sec={tv_sec}, tv_usec={tv_usec}");
state.on_copy_complete(tv_sec, tv_usec);
}
ScreencopyFrameEvent::Failed => {
tracing::error!("Screencopy frame failed");
state.on_copy_fail();
}
ScreencopyFrameEvent::Damage { .. } => {}
_ => {}
}
}
}
impl<S: crate::state::CaptureSource> Dispatch<ZwlrScreencopyManagerV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwlrScreencopyManagerV1,
_event: <ZwlrScreencopyManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
+133
View File
@@ -0,0 +1,133 @@
use wayland_client::protocol::wl_output::WlOutput;
use wayland_client::{Dispatch, Proxy, QueueHandle};
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_v1::{
Event as XdgOutputEvent, ZxdgOutputV1,
};
use crate::state::{CaptureSource, EncConstructionStage, OutputId, State, Transform};
impl<S: CaptureSource> Dispatch<WlOutput, OutputId> for State<S> {
fn event(
state: &mut Self,
_proxy: &WlOutput,
event: wayland_client::protocol::wl_output::Event,
data: &OutputId,
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
use wayland_client::protocol::wl_output::Event as OutputEvent;
use wayland_client::protocol::wl_output::Mode as WlMode;
use wayland_client::protocol::wl_output::Transform as WlTransform;
let OutputId(target_name) = data;
let idx = match &state.stage {
EncConstructionStage::ProbingOutputs { output_names, .. } => {
output_names.iter().position(|&n| n == *target_name)
}
_ => None,
};
let idx = match idx {
Some(i) => i,
None => return,
};
match event {
OutputEvent::Geometry { transform, .. } => {
let t = match transform {
wayland_client::WEnum::Value(WlTransform::Normal) => Transform::Normal,
wayland_client::WEnum::Value(WlTransform::_90) => Transform::Normal90,
wayland_client::WEnum::Value(WlTransform::_180) => Transform::Normal180,
wayland_client::WEnum::Value(WlTransform::_270) => Transform::Normal270,
wayland_client::WEnum::Value(WlTransform::Flipped) => Transform::Flipped,
wayland_client::WEnum::Value(WlTransform::Flipped90) => Transform::Flipped90,
wayland_client::WEnum::Value(WlTransform::Flipped180) => Transform::Flipped180,
wayland_client::WEnum::Value(WlTransform::Flipped270) => Transform::Flipped270,
_ => Transform::Normal,
};
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.transform = Some(t);
}
}
}
OutputEvent::Mode {
width,
height,
flags,
..
} => {
let is_current = matches!(flags, wayland_client::WEnum::Value(WlMode::Current));
if is_current {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.mode_size = Some((width, height));
}
}
}
}
OutputEvent::Done => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.done_count += 1;
if info.done_count >= 1 {
state.try_finalize_output(idx);
}
}
}
}
OutputEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.wl_name = Some(name);
}
}
}
_ => {}
}
}
}
impl<S: CaptureSource> Dispatch<ZxdgOutputV1, OutputId> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZxdgOutputV1,
event: <ZxdgOutputV1 as Proxy>::Event,
data: &OutputId,
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
let target_name = data.0;
let idx = match &state.stage {
EncConstructionStage::ProbingOutputs { output_names, .. } => {
output_names.iter().position(|&n| n == target_name)
}
_ => None,
};
let idx = match idx {
Some(i) => i,
None => return,
};
match event {
XdgOutputEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.name = Some(name);
}
}
}
XdgOutputEvent::LogicalSize { .. } => {}
XdgOutputEvent::Done => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.done_count += 1;
if info.done_count >= 1 {
state.try_finalize_output(idx);
}
}
}
}
_ => {}
}
}
}
+57 -705
View File
@@ -9,34 +9,16 @@ use std::time::Instant;
use anyhow::Result;
use wayland_client::backend::ObjectId;
use wayland_client::globals::{GlobalList, GlobalListContents};
use wayland_client::globals::GlobalList;
use wayland_client::protocol::wl_buffer::WlBuffer;
use wayland_client::protocol::wl_output::WlOutput;
use wayland_client::protocol::wl_registry::WlRegistry;
use wayland_client::{event_created_child, Dispatch, Proxy, QueueHandle};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_buffer_params_v1::{
Event as BufferParamsEvent, Flags as BufferParamsFlags, ZwpLinuxBufferParamsV1,
};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_feedback_v1::{
Event as DmabufFeedbackEvent, ZwpLinuxDmabufFeedbackV1,
};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_v1::{
Event as DmabufEvent, ZwpLinuxDmabufV1,
};
use wayland_client::{Dispatch, QueueHandle};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_buffer_params_v1::Flags as BufferParamsFlags;
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_feedback_v1::ZwpLinuxDmabufFeedbackV1;
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_v1::ZwpLinuxDmabufV1;
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_manager_v1::ZxdgOutputManagerV1;
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_v1::{
Event as XdgOutputEvent, ZxdgOutputV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_head_v1::{
self, Event as WlrHeadEvent, ZwlrOutputHeadV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_manager_v1::{
self, Event as WlrOutputManagerEvent, ZwlrOutputManagerV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_mode_v1::ZwlrOutputModeV1;
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_frame_v1::{
Event as ScreencopyFrameEvent, ZwlrScreencopyFrameV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_manager_v1::ZwlrOutputManagerV1;
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_frame_v1::ZwlrScreencopyFrameV1;
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_manager_v1::ZwlrScreencopyManagerV1;
use ffmpeg_next as ff;
@@ -44,12 +26,13 @@ use ffmpeg_next::ffi;
use crate::args::Args;
use crate::avhw::{AvHwDevCtx, EncState, EncodedH264Frame, SwEncState};
use crate::cap_wlr_screencopy::CapWlrScreencopy;
use crate::fps_limit::FpsLimit;
use crate::stats::{FrameTimings, PipelineStats};
use crate::transform::{transpose_if_transform_transposed, Transform};
use crate::webrtc::WebRtcState;
mod dispatch;
// ---------------------------------------------------------------------------
// CaptureSource trait
// ---------------------------------------------------------------------------
@@ -65,8 +48,6 @@ pub trait CaptureSource: Sized + 'static {
qh: &QueueHandle<State<Self>>,
) -> Result<Self>;
fn alloc_frame(&mut self) -> Option<Self::Frame>;
fn queue_copy(&mut self, buffer: &WlBuffer, qh: &QueueHandle<State<Self>>);
fn on_done_with_frame(&mut self, frame: Self::Frame);
@@ -79,40 +60,24 @@ pub trait CaptureSource: Sized + 'static {
pub struct OutputInfo {
pub name: String,
pub transform: Transform,
pub physical_size: (i32, i32),
pub logical_position: (i32, i32),
}
#[derive(Default)]
pub struct PartialOutputInfo {
pub name: Option<String>,
/// Name from wl_output::Name (v4) — used to match wlr-output-management heads
pub wl_name: Option<String>,
pub transform: Option<Transform>,
pub physical_size: Option<(i32, i32)>,
pub logical_position: Option<(i32, i32)>,
// Pixel dimensions from Mode event — preparatory for Phase 2 resolution logic
pub mode_size: Option<(i32, i32)>,
pub done_count: u32,
}
impl Default for PartialOutputInfo {
fn default() -> Self {
Self {
name: None,
wl_name: None,
transform: None,
physical_size: None,
logical_position: None,
mode_size: None,
done_count: 0,
}
}
}
/// Stores head info from wlr-output-management for name-based matching with wl_output.
struct WlrHeadInfo {
position: Option<(i32, i32)>,
}
/// Marker for wlr-output-management heads seen during probing; tracked by name
/// in `EncConstructionStage::ProbingOutputs.wlr_heads`.
// `pub(crate)` (not module-private): exposed via `EncConstructionStage::ProbingOutputs.wlr_heads`
// which is reached from main.rs during the wlr-screencopy probing loop.
pub(crate) struct WlrHeadInfo {}
/// User data for XdgOutput dispatch to identify which WlOutput it belongs to.
pub struct OutputId(pub u32);
@@ -138,7 +103,10 @@ impl StreamingEncoder {
}
}
fn encode_frame(&mut self, hw_frame: &ffmpeg_next::frame::Video) -> anyhow::Result<()> {
fn encode_frame(
&mut self,
hw_frame: &ffmpeg_next::frame::Video,
) -> anyhow::Result<crate::avhw::EncodeStages> {
match self {
StreamingEncoder::Mp4(enc) => enc.encode_frame(hw_frame),
StreamingEncoder::WebRtc(enc) => enc.encode_frame(hw_frame),
@@ -156,8 +124,14 @@ impl StreamingEncoder {
// ---------------------------------------------------------------------------
// EncConstructionStage
// ---------------------------------------------------------------------------
//
// `pub(crate)` (not `pub`): this enum leaks the private `WlrHeadInfo` type via
// its `wlr_heads` field, and the construction-stage state machine is an
// internal implementation detail. Crate-internal consumers (main.rs) get there
// via `crate::state::`; there is no need to expose this across the crate
// boundary. See Oracle audit 2026-06-28.
pub enum EncConstructionStage<S: CaptureSource> {
pub(crate) enum EncConstructionStage<S: CaptureSource> {
ProbingOutputs {
outputs: Vec<PartialOutputInfo>,
bound_outputs: Vec<WlOutput>,
@@ -180,7 +154,6 @@ pub enum EncConstructionStage<S: CaptureSource> {
dmabuf: ZwpLinuxDmabufV1,
},
Streaming {
output_info: OutputInfo,
output: WlOutput,
enc: StreamingEncoder,
cap: S,
@@ -197,10 +170,12 @@ pub enum EncConstructionStage<S: CaptureSource> {
pub enum InFlightSurface<S: CaptureSource> {
None,
AllocQueued,
Allocd(S::Frame),
CopyQueued {
surface: ff::frame::Video,
drm_map: ff::ffi::AVDRMFrameDescriptor,
// Boxed: AVDRMFrameDescriptor is ~592 bytes (4 objects + 4 layers),
// which would balloon every InFlightSurface variant via enum alignment.
// The box shrinks the enum to ~32 bytes regardless of variant.
drm_map: Box<ff::ffi::AVDRMFrameDescriptor>,
frame: S::Frame,
buffer: WlBuffer,
},
@@ -211,9 +186,8 @@ pub enum InFlightSurface<S: CaptureSource> {
// ---------------------------------------------------------------------------
pub struct State<S: CaptureSource> {
pub stage: EncConstructionStage<S>,
pub(crate) stage: EncConstructionStage<S>,
pub in_flight_surface: InFlightSurface<S>,
pub starting_timestamp: Option<i64>,
pub stats_start_time: Option<Instant>,
pub stats_last_time: Option<Instant>,
pub stats_frames: u64,
@@ -306,7 +280,6 @@ impl<S: CaptureSource> State<S> {
wlr_head_proxy_to_name: HashMap::new(),
},
in_flight_surface: InFlightSurface::None,
starting_timestamp: None,
stats_start_time: None,
stats_last_time: None,
stats_frames: 0,
@@ -483,7 +456,6 @@ impl<S: CaptureSource> State<S> {
pub fn on_frame_allocd(&mut self, frame: S::Frame, format: u32, width: u32, height: u32) {
let (frames_rgb_ctx, dmabuf, cap) = match &mut self.stage {
EncConstructionStage::Streaming {
output_info: _,
output: _,
enc,
dmabuf,
@@ -512,6 +484,10 @@ impl<S: CaptureSource> State<S> {
unsafe {
(*map_frame.as_mut_ptr()).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME as i32;
}
// SAFETY: map_frame and surface are valid, owned AVFrame pointers from
// av_hwframe_get/surface.alloc above. AV_HWFRAME_MAP_READ flag (0 here)
// requests a read-only mapping. The DRM_PRIME format set above instructs
// FFmpeg to populate data[0] with an AVDRMFrameDescriptor on success.
let ret = unsafe { ffi::av_hwframe_map(map_frame.as_mut_ptr(), surface.as_ptr(), 0) };
if ret < 0 {
tracing::error!("av_hwframe_map failed: {}", crate::avhw::ff_err(ret));
@@ -571,7 +547,7 @@ impl<S: CaptureSource> State<S> {
);
self.in_flight_surface = InFlightSurface::CopyQueued {
surface,
drm_map: desc,
drm_map: Box::new(desc),
frame,
buffer: wl_buffer,
};
@@ -629,15 +605,22 @@ impl<S: CaptureSource> State<S> {
};
if should_encode {
let encode_start = Instant::now();
if let Err(e) = enc.encode_frame(&surface) {
tracing::error!("encode_frame failed: {}", e);
self.errored = true;
match enc.encode_frame(&surface) {
Ok(stages) => {
let encode_elapsed = encode_start.elapsed().as_micros() as u64;
self.stats.record_encode(&FrameTimings {
scale_us: stages.scale_us,
transfer_us: stages.transfer_us,
encode_us: stages.encode_us,
total_us: encode_elapsed,
..Default::default()
});
}
Err(e) => {
tracing::error!("encode_frame failed: {}", e);
self.errored = true;
}
}
let encode_elapsed = encode_start.elapsed().as_micros() as u64;
self.stats.record_encode(&FrameTimings {
total_us: encode_elapsed,
..Default::default()
});
}
self.stats_frames += 1;
if let Some(last) = self.stats_last_time {
@@ -706,9 +689,7 @@ impl<S: CaptureSource> State<S> {
continue;
}
count += 1;
if let Err(e) = wrtc
.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks)
{
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks) {
tracing::debug!("WebRTC write frame error: {e}");
}
self.stats.record_send(0.0, None);
@@ -822,7 +803,6 @@ impl<S: CaptureSource> State<S> {
bitrate
);
self.stage = EncConstructionStage::Streaming {
output_info,
output,
enc,
cap,
@@ -832,22 +812,6 @@ impl<S: CaptureSource> State<S> {
}
fn try_finalize_output(&mut self, _idx: usize) -> bool {
// Merge wlr head position info into outputs (needed for niri path)
if let EncConstructionStage::ProbingOutputs {
outputs, wlr_heads, ..
} = &mut self.stage
{
for info in outputs.iter_mut() {
if info.logical_position.is_none() {
if let Some(ref wl_name) = info.wl_name {
if let Some(head_info) = wlr_heads.get(wl_name) {
info.logical_position = head_info.position;
}
}
}
}
}
let (target_idx, output_count) = match &self.stage {
EncConstructionStage::ProbingOutputs {
outputs,
@@ -890,24 +854,20 @@ impl<S: CaptureSource> State<S> {
Some(i) => {
let info = &outputs[i];
if has_xdg {
// xdg-output path (Sway/Hyprland) — strict checks
// done_count >= 2 implies physical_size and logical_position
// already arrived (Wayland: Geometry/Mode/Position fire before Done).
if info.done_count < 2
|| info.name.is_none()
|| info.transform.is_none()
|| info.physical_size.is_none()
|| info.logical_position.is_none()
{
return false;
}
} else {
// wlr-output-management path (niri) — relaxed checks
if info.done_count < 1 || !wlr_manager_done {
// done_count >= 1 implies transform arrived (Geometry precedes Done).
if info.done_count < 1 || !wlr_manager_done || info.transform.is_none()
{
return false;
}
if info.transform.is_none() || info.physical_size.is_none() {
return false;
}
// name and logical_position can use defaults
}
(i, output_count)
}
@@ -978,8 +938,6 @@ impl<S: CaptureSource> State<S> {
.or(info.wl_name.clone())
.unwrap_or_else(|| format!("output-{}", output_names[target_idx])),
transform: info.transform.unwrap(),
physical_size: info.physical_size.unwrap(),
logical_position: info.logical_position.unwrap_or((0, 0)),
};
let output = bound_outputs[target_idx].clone();
@@ -1039,609 +997,3 @@ impl<S: CaptureSource> State<S> {
true
}
}
// ---------------------------------------------------------------------------
// Dispatch<WlRegistry, GlobalListContents>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<WlRegistry, GlobalListContents> for State<S> {
fn event(
state: &mut Self,
registry: &WlRegistry,
event: wayland_client::protocol::wl_registry::Event,
_data: &GlobalListContents,
_conn: &wayland_client::Connection,
qhandle: &QueueHandle<State<S>>,
) {
use wayland_client::protocol::wl_registry::Event as RegistryEvent;
match event {
RegistryEvent::Global {
name,
interface,
version,
} => match interface.as_str() {
"zwlr_screencopy_manager_v1" => {
let v = version.min(3);
tracing::debug!("Binding zwlr_screencopy_manager_v1 v{v} (name={name})");
let mgr: ZwlrScreencopyManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
screencopy_manager, ..
} = &mut state.stage
{
*screencopy_manager = Some(mgr);
}
}
"zwp_linux_dmabuf_v1" => {
let v = version.min(4);
tracing::debug!("Binding zwp_linux_dmabuf_v1 v{v} (name={name})");
let proxy: ZwpLinuxDmabufV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
dmabuf,
dmabuf_feedback,
..
} = &mut state.stage
{
*dmabuf = Some(proxy.clone());
if v >= 4 {
let feedback = proxy.get_default_feedback(qhandle, ());
*dmabuf_feedback = Some(feedback);
}
}
}
"wl_output" => {
let v = version.min(4);
tracing::debug!("Binding wl_output v{v} (name={name})");
let output: WlOutput = registry.bind(name, v, qhandle, OutputId(name));
if let EncConstructionStage::ProbingOutputs {
outputs,
bound_outputs,
output_names,
xdg_output_manager,
..
} = &mut state.stage
{
outputs.push(PartialOutputInfo::default());
bound_outputs.push(output.clone());
output_names.push(name);
if let Some(xdg_mgr) = xdg_output_manager {
let output_id = OutputId(name);
xdg_mgr.get_xdg_output(&output, qhandle, output_id);
}
}
}
"zxdg_output_manager_v1" => {
let v = version.min(3);
tracing::debug!("Binding zxdg_output_manager_v1 v{v} (name={name})");
let xdg_mgr: ZxdgOutputManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
bound_outputs,
xdg_output_manager,
output_names,
..
} = &mut state.stage
{
for (i, output) in bound_outputs.iter().enumerate() {
let oname = output_names.get(i).copied().unwrap_or(0);
let output_id = OutputId(oname);
xdg_mgr.get_xdg_output(output, qhandle, output_id);
}
*xdg_output_manager = Some(xdg_mgr);
}
}
"zwlr_output_manager_v1" => {
let v = version.min(4);
tracing::debug!("Binding zwlr_output_manager_v1 v{v} (name={name})");
let mgr: ZwlrOutputManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
wlr_output_manager, ..
} = &mut state.stage
{
*wlr_output_manager = Some(mgr);
}
}
_ => {}
},
RegistryEvent::GlobalRemove { name } => {
tracing::debug!("Global removed: name={name}");
}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<WlOutput, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<WlOutput, OutputId> for State<S> {
fn event(
state: &mut Self,
_proxy: &WlOutput,
event: wayland_client::protocol::wl_output::Event,
data: &OutputId,
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
use wayland_client::protocol::wl_output::Event as OutputEvent;
use wayland_client::protocol::wl_output::Mode as WlMode;
use wayland_client::protocol::wl_output::Transform as WlTransform;
let OutputId(target_name) = data;
let idx = match &state.stage {
EncConstructionStage::ProbingOutputs { output_names, .. } => {
output_names.iter().position(|&n| n == *target_name)
}
_ => None,
};
let idx = match idx {
Some(i) => i,
None => return,
};
match event {
OutputEvent::Geometry {
transform,
physical_width,
physical_height,
..
} => {
let t = match transform {
wayland_client::WEnum::Value(WlTransform::Normal) => Transform::Normal,
wayland_client::WEnum::Value(WlTransform::_90) => Transform::Normal90,
wayland_client::WEnum::Value(WlTransform::_180) => Transform::Normal180,
wayland_client::WEnum::Value(WlTransform::_270) => Transform::Normal270,
wayland_client::WEnum::Value(WlTransform::Flipped) => Transform::Flipped,
wayland_client::WEnum::Value(WlTransform::Flipped90) => Transform::Flipped90,
wayland_client::WEnum::Value(WlTransform::Flipped180) => Transform::Flipped180,
wayland_client::WEnum::Value(WlTransform::Flipped270) => Transform::Flipped270,
_ => Transform::Normal,
};
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.transform = Some(t);
info.physical_size = Some((physical_width, physical_height));
}
}
}
OutputEvent::Mode {
width,
height,
flags,
..
} => {
let is_current = matches!(flags, wayland_client::WEnum::Value(WlMode::Current));
if is_current {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.mode_size = Some((width, height));
}
}
}
}
OutputEvent::Done => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.done_count += 1;
if info.done_count >= 1 {
state.try_finalize_output(idx);
}
}
}
}
OutputEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.wl_name = Some(name);
}
}
}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZxdgOutputV1, OutputId>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZxdgOutputV1, OutputId> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZxdgOutputV1,
event: <ZxdgOutputV1 as Proxy>::Event,
data: &OutputId,
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
let target_name = data.0;
let idx = match &state.stage {
EncConstructionStage::ProbingOutputs { output_names, .. } => {
output_names.iter().position(|&n| n == target_name)
}
_ => None,
};
let idx = match idx {
Some(i) => i,
None => return,
};
match event {
XdgOutputEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.name = Some(name);
}
}
}
XdgOutputEvent::LogicalPosition { x, y } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.logical_position = Some((x, y));
}
}
}
XdgOutputEvent::LogicalSize { .. } => {}
XdgOutputEvent::Done => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.done_count += 1;
if info.done_count >= 1 {
state.try_finalize_output(idx);
}
}
}
}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwpLinuxDmabufV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwpLinuxDmabufV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwpLinuxDmabufV1,
event: <ZwpLinuxDmabufV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
DmabufEvent::Format { .. } => {}
DmabufEvent::Modifier { .. } => {}
_ => {}
}
}
}
impl<S: CaptureSource> Dispatch<ZwpLinuxDmabufFeedbackV1, ()> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZwpLinuxDmabufFeedbackV1,
event: <ZwpLinuxDmabufFeedbackV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
DmabufFeedbackEvent::MainDevice { device } => {
if device.len() >= 8 {
let dev_bytes: [u8; 8] = device[..8].try_into().unwrap_or([0u8; 8]);
let dev = u64::from_ne_bytes(dev_bytes);
let minor = ((dev & 0xFF) | ((dev >> 12) & 0xFFFFFF00)) as u32;
let path = PathBuf::from(format!("/dev/dri/renderD{}", minor));
if path.exists() {
tracing::info!(
"Compositor DRM device: {} (dev_t: {})",
path.display(),
dev
);
state.drm_device_from_compositor = Some(path);
} else {
tracing::warn!(
"Compositor reported DRM device {} (dev_t: {}) but path does not exist",
path.display(),
dev
);
}
} else {
tracing::warn!(
"main_device event with unexpected data length: {}",
device.len()
);
}
}
DmabufFeedbackEvent::FormatTable { .. } => {}
DmabufFeedbackEvent::Done => {}
DmabufFeedbackEvent::TrancheDone => {}
DmabufFeedbackEvent::TrancheTargetDevice { .. } => {}
DmabufFeedbackEvent::TrancheFormats { .. } => {}
DmabufFeedbackEvent::TrancheFlags { .. } => {}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwpLinuxBufferParamsV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwpLinuxBufferParamsV1, ()> for State<S> {
fn event(
state: &mut Self,
proxy: &ZwpLinuxBufferParamsV1,
event: <ZwpLinuxBufferParamsV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
BufferParamsEvent::Created { .. } => {
tracing::debug!("DMA-BUF buffer created");
}
BufferParamsEvent::Failed => {
tracing::error!("DMA-BUF buffer creation failed");
let taken = mem::replace(&mut state.in_flight_surface, InFlightSurface::None);
match taken {
InFlightSurface::CopyQueued { buffer, frame, .. } => {
drop(buffer);
if let EncConstructionStage::Streaming { cap, .. } = &mut state.stage {
cap.on_done_with_frame(frame);
}
}
other => {
state.in_flight_surface = other;
}
}
proxy.destroy();
state.errored = true;
}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrScreencopyFrameV1, ()> for CapWlrScreencopy
// ---------------------------------------------------------------------------
impl Dispatch<ZwlrScreencopyFrameV1, ()> for State<CapWlrScreencopy> {
fn event(
state: &mut Self,
proxy: &ZwlrScreencopyFrameV1,
event: <ZwlrScreencopyFrameV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<CapWlrScreencopy>>,
) {
match event {
// SHM buffer offer — in v3 the compositor enumerates supported buffer
// types (buffer and/or linux_dmabuf) before buffer_done. We only
// support DMA-BUF, so just log and wait for linux_dmabuf / buffer_done.
ScreencopyFrameEvent::Buffer { .. } => {
tracing::debug!("Received SHM Buffer offer — only DMA-BUF capture is supported");
}
ScreencopyFrameEvent::LinuxDmabuf {
format,
width,
height,
} => {
tracing::debug!("Screencopy LinuxDmabuf: format={format}, {width}x{height}");
if !matches!(state.in_flight_surface, InFlightSurface::AllocQueued) {
tracing::warn!("Received LinuxDmabuf while no frame allocation was queued");
return;
}
if matches!(state.stage, EncConstructionStage::EverythingButFmt { .. }) {
state.negotiate_format(format, width, height);
if state.errored {
return;
}
}
if let EncConstructionStage::Streaming { cap, .. } = &mut state.stage {
cap.current_frame = Some(proxy.clone());
}
state.on_frame_allocd((), format, width, height);
}
// v3 terminal event: all buffer offers have been enumerated.
// If still AllocQueued, the compositor never sent linux_dmabuf —
// DMA-BUF screencopy is unsupported, so we must error out.
ScreencopyFrameEvent::BufferDone => {
if matches!(state.in_flight_surface, InFlightSurface::AllocQueued) {
tracing::error!(
"Compositor did not offer DMA-BUF screencopy (only SHM); \
DMA-BUF capture is required"
);
state.in_flight_surface = InFlightSurface::None;
proxy.destroy();
state.errored = true;
}
}
ScreencopyFrameEvent::Ready {
tv_sec_hi,
tv_sec_lo,
tv_nsec,
} => {
let tv_sec = (tv_sec_hi as u64) << 32 | tv_sec_lo as u64;
let tv_usec = tv_nsec / 1000;
tracing::trace!("Screencopy ready: tv_sec={tv_sec}, tv_usec={tv_usec}");
state.on_copy_complete(tv_sec, tv_usec);
}
ScreencopyFrameEvent::Failed => {
tracing::error!("Screencopy frame failed");
state.on_copy_fail();
}
ScreencopyFrameEvent::Damage { .. } => {}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZxdgOutputManagerV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZxdgOutputManagerV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZxdgOutputManagerV1,
_event: <ZxdgOutputManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrOutputManagerV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwlrOutputManagerV1, ()> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZwlrOutputManagerV1,
event: <ZwlrOutputManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
qhandle: &QueueHandle<State<S>>,
) {
match event {
WlrOutputManagerEvent::Head { head } => {
let _head: ZwlrOutputHeadV1 = head;
tracing::debug!("wlr output head advertised");
}
WlrOutputManagerEvent::Done { .. } => {
if let EncConstructionStage::ProbingOutputs {
wlr_manager_done,
outputs,
..
} = &mut state.stage
{
*wlr_manager_done = true;
let count = outputs.len();
for idx in 0..count {
state.try_finalize_output(idx);
}
}
}
WlrOutputManagerEvent::Finished { .. } => {
tracing::warn!("zwlr_output_manager_v1::Finished received during probing");
}
_ => {}
}
}
event_created_child!(State<S>, ZwlrOutputManagerV1, [
zwlr_output_manager_v1::EVT_HEAD_OPCODE => (ZwlrOutputHeadV1, ()),
]);
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrOutputHeadV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwlrOutputHeadV1, ()> for State<S> {
fn event(
state: &mut Self,
proxy: &ZwlrOutputHeadV1,
event: <ZwlrOutputHeadV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
WlrHeadEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs {
wlr_heads,
wlr_head_proxy_to_name,
..
} = &mut state.stage
{
wlr_heads
.entry(name.clone())
.or_insert(WlrHeadInfo { position: None });
wlr_head_proxy_to_name.insert(proxy.id(), name);
}
}
WlrHeadEvent::Position { x, y } => {
if let EncConstructionStage::ProbingOutputs {
wlr_heads,
wlr_head_proxy_to_name,
..
} = &mut state.stage
{
if let Some(name) = wlr_head_proxy_to_name.get(&proxy.id()) {
if let Some(head) = wlr_heads.get_mut(name) {
head.position = Some((x, y));
}
}
}
}
WlrHeadEvent::Finished { .. } => {
tracing::debug!("zwlr_output_head_v1::Finished received");
}
_ => {}
}
}
event_created_child!(State<S>, ZwlrOutputHeadV1, [
zwlr_output_head_v1::EVT_MODE_OPCODE => (ZwlrOutputModeV1, ()),
]);
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrOutputModeV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwlrOutputModeV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwlrOutputModeV1,
_event: <ZwlrOutputModeV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrScreencopyManagerV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwlrScreencopyManagerV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwlrScreencopyManagerV1,
_event: <ZwlrScreencopyManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
// ---------------------------------------------------------------------------
// Dispatch<WlBuffer, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<WlBuffer, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &WlBuffer,
event: <WlBuffer as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
if let wayland_client::protocol::wl_buffer::Event::Release = event {
tracing::trace!("WlBuffer released");
}
}
}
+68 -437
View File
@@ -1,4 +1,8 @@
// 采集门户状态模块 —— 通过 PipeWire/DMA-BUF 进行屏幕采集并编码
// AsRawFd is required by frame.fd.as_raw_fd() in build_drm_descriptor below
// but rustc emits a false "unused_imports" warning because OwnedFd also has
// an inherent as_raw_fd — same quirk as avhw.rs. E0599 if removed → keep it.
#[allow(unused_imports)]
use std::os::fd::AsRawFd;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
@@ -9,13 +13,21 @@ use anyhow::{bail, Result}; // 错误处理工具
use crate::args::Args; // 命令行参数
use crate::avhw::{
self, BitrateCommand, CpuNv12Frame, EncodeOutcome, EncodedH264Frame, ResolutionChange,
SwEncEncode, SwEncImport, SwEncState,
self, BitrateCommand, CpuNv12Frame, ResolutionChange, SwEncEncode, SwEncImport, SwEncState,
}; // 软件编码器状态(VAAPI 导入 + H.264 编码)
use crate::cap_portal::{CapPortal, PwCtrlEvent, PwDmaBufFrame}; // PipeWire 屏幕采集端点
use crate::stats::{FrameTimings, PipelineStats}; // 管道统计(帧计时、每秒快照)
use crate::webrtc::WebRtcState; // WebRTC 信令与媒体传输
mod bitrate;
use bitrate::webrtc_startup_bitrate_bps;
mod threads;
use threads::{
encode_thread_loop, webrtc_thread_loop, EncodeThread, EncodeThreadTiming, WebRtcThreadChannels,
WebRtcThreadConfig, WebrtcThread,
};
/// 门户采集的阶段状态
/// - WaitingForFormat: 等待接收到第一帧 DMA-BUF 以确定视频格式参数
/// - Streaming: 已完成初始化,正在持续编码流
@@ -24,24 +36,6 @@ enum PortalStage {
Streaming,
}
struct EncodeThreadTiming {
sws_us: u64,
encode_us: u64,
output_bytes: usize,
}
struct EncodeThread {
handle: Option<std::thread::JoinHandle<()>>,
input_tx: crossbeam_channel::Sender<CpuNv12Frame>,
timing_rx: crossbeam_channel::Receiver<EncodeThreadTiming>,
duplicate_count: std::sync::Arc<std::sync::atomic::AtomicU64>,
}
struct WebrtcThread {
handle: Option<std::thread::JoinHandle<()>>,
sent_gap_rx: crossbeam_channel::Receiver<(f64, Option<f64>)>,
}
/// 门户模式的主状态机
///
/// 负责管理从 PipeWire 采集屏幕帧、通过 VAAPI 硬件编码的完整生命周期。
@@ -249,9 +243,7 @@ impl StatePortal {
bitrate_rx,
encoder_resolution_rx,
)?;
let duplicate_count = std::sync::Arc::new(
std::sync::atomic::AtomicU64::new(0),
);
let duplicate_count = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0));
let duplicate_count_for_thread = duplicate_count.clone();
let handle = std::thread::Builder::new()
.name("wl-webrtc-encode".into())
@@ -288,15 +280,19 @@ impl StatePortal {
.spawn(move || {
webrtc_thread_loop(
wrtc,
webrtc_rx,
fps,
enc_width,
enc_height,
max_bitrate,
WebRtcThreadConfig {
fps,
enc_width,
enc_height,
max_bitrate,
},
WebRtcThreadChannels {
webrtc_rx,
sent_gap_tx,
bitrate_tx,
resolution_tx,
},
paused,
sent_gap_tx,
bitrate_tx,
resolution_tx,
)
})?;
self.webrtc_thread = Some(WebrtcThread {
@@ -338,8 +334,17 @@ impl StatePortal {
// 每秒输出一次结构化管道统计(仅 --stats 启用时记录日志)
if self.args.stats && self.stats.should_snapshot() {
self.stats.set_pipewire_dropped(0, 0);
self.stats.set_queue_depths(0, 0);
// Wire PipeWire drop counter (delta-tracked via pw_dropped_prev) and
// capture channel depth. Oracle audit 2026-06-28: previously hardcoded
// (0, 0), which silently zeroed two real diagnostic fields.
let total_dropped = self.cap.dropped_count();
self.stats
.set_pipewire_dropped(total_dropped, self.pw_dropped_prev);
self.pw_dropped_prev = total_dropped;
// capture queue depth is real; encoded side has no exposed depth — the
// encoder thread publishes timings only, not a frame queue length.
self.stats
.set_queue_depths(self.cap.capture_queue_depth(), 0);
if let Some(ref enc_thread) = self.enc_thread {
while let Ok(timing) = enc_thread.timing_rx.try_recv() {
self.stats.record_encode_thread(
@@ -478,55 +483,45 @@ impl StatePortal {
if let Some(enc) = self.enc.as_mut() {
// 将 DMA-BUF 帧零拷贝导入 VAAPI 硬件帧池
let mut vaapi_frame = unsafe {
avhw::import_dma_buf_to_vaapi(
enc.frames_rgb().as_ptr(),
frame.fd.as_raw_fd(),
frame.width,
frame.height,
frame.format,
frame.modifier,
frame.stride,
frame.offset,
)
}?;
// SAFETY: delegates to avhw::import_dma_buf_to_vaapi (itself an unsafe fn);
// frames_rgb pointer is a valid AVBufferRef owned by enc, and `frame` is the
// PipeWire-formatted PwDmaBufFrame whose metadata the function reads directly.
// See that function's own SAFETY contract.
let mut vaapi_frame =
unsafe { avhw::import_dma_buf_to_vaapi(enc.frames_rgb().as_ptr(), &frame) }?;
let import_us = t_import_start.elapsed().as_micros() as u64;
let t_encode_start = Instant::now();
// 设置帧的显示时间戳(PTS),基于已编码帧序号
// SAFETY: vaapi_frame is the freshly-imported valid AVFrame returned by
// import_dma_buf_to_vaapi above; pts is a plain i64 field on AVFrame.
unsafe {
(*vaapi_frame.as_mut_ptr()).pts = pts;
}
// 送入编码器完成:缩放 → 回读 → 格式转换 → H.264 编码
enc.encode_frame(&vaapi_frame)?;
let stages = enc.encode_frame(&vaapi_frame)?;
let total_us = t_import_start.elapsed().as_micros() as u64;
let encode_us = t_encode_start.elapsed().as_micros() as u64;
let encode_us = stages.encode_us;
self.frames_encoded += 1;
// 记录帧计时到管道统计(import + encode 内部各阶段暂不可分离,用 total 覆盖
// 记录帧计时到管道统计(scale 来自 filter graphtransfer 在 HW 路径恒为 0
let timings = FrameTimings {
import_us,
scale_us: stages.scale_us,
transfer_us: stages.transfer_us,
encode_us,
total_us,
..Default::default()
};
self.stats.record_encode(&timings);
} else if let Some(import) = self.enc_import.as_mut() {
let mut vaapi_frame = unsafe {
avhw::import_dma_buf_to_vaapi(
import.frames_rgb().as_ptr(),
frame.fd.as_raw_fd(),
frame.width,
frame.height,
frame.format,
frame.modifier,
frame.stride,
frame.offset,
)
}?;
// SAFETY: same contract as the enc branch above — frames_rgb owned by
// import, `frame` carries the PipeWire DMA-BUF metadata.
let mut vaapi_frame =
unsafe { avhw::import_dma_buf_to_vaapi(import.frames_rgb().as_ptr(), &frame) }?;
// SAFETY: vaapi_frame is the valid AVFrame returned above; pts is plain i64.
unsafe {
(*vaapi_frame.as_mut_ptr()).pts = pts;
}
@@ -652,253 +647,6 @@ impl StatePortal {
}
}
fn encode_thread_loop(
mut encode: SwEncEncode,
input_rx: crossbeam_channel::Receiver<CpuNv12Frame>,
timing_tx: crossbeam_channel::Sender<EncodeThreadTiming>,
duplicate_count: std::sync::Arc<std::sync::atomic::AtomicU64>,
) {
loop {
match input_rx.recv() {
Ok(frame) => {
match encode.encode_cpu_frame(&frame) {
Ok(EncodeOutcome::Encoded) => {
let t = encode.take_timing();
let _ = timing_tx.try_send(EncodeThreadTiming {
sws_us: t.sws_us,
encode_us: t.encode_us,
output_bytes: t.output_bytes,
});
}
Ok(EncodeOutcome::SkippedDuplicate) => {
duplicate_count.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
}
Ok(_) => {
// SkippedPaused / SkippedDisconnected — no counter needed
}
Err(e) => {
tracing::error!("Encode thread error: {e}");
break;
}
}
}
Err(_) => {
tracing::info!("Encode thread input closed, flushing encoder");
if let Err(e) = encode.flush() {
tracing::error!("Encode thread flush error: {e}");
}
break;
}
}
}
tracing::info!("Encode thread exiting");
}
fn webrtc_thread_loop(
mut wrtc: WebRtcState,
webrtc_rx: crossbeam_channel::Receiver<EncodedH264Frame>,
fps: u32,
enc_width: u32,
enc_height: u32,
max_bitrate: u64,
paused: Arc<AtomicBool>,
sent_gap_tx: crossbeam_channel::Sender<(f64, Option<f64>)>,
bitrate_tx: crossbeam_channel::Sender<BitrateCommand>,
resolution_tx: crossbeam_channel::Sender<BitrateCommand>,
) {
let mut frames_sent: u64 = 0;
let mut last_send: Option<std::time::Instant> = None;
let mut last_sent_bitrate: Option<u64> = None;
let initial_tier = (enc_width, enc_height);
let mut current_tier = initial_tier;
let mut upscale_counter = 0u32;
let mut last_resolution_eval = Instant::now();
let timeout = Duration::from_millis(1);
loop {
if let Err(e) = wrtc.handle_signaling() {
tracing::error!("WebRTC signaling error: {e}");
break;
}
if let Err(e) = wrtc.poll_and_feed() {
tracing::error!("WebRTC poll error: {e}");
break;
}
if wrtc.take_force_keyframe() {
let _ = bitrate_tx.try_send(BitrateCommand::ForceKeyframe);
}
let connected = wrtc.is_connected();
let was_paused = paused.load(Ordering::Relaxed);
let now_paused = !connected;
if was_paused && !now_paused {
tracing::info!("WebRTC client connected, resuming encoding");
} else if !was_paused && now_paused {
tracing::warn!("WebRTC client disconnected, pausing encoding");
}
paused.store(now_paused, Ordering::Relaxed);
if let Some(bwe) = wrtc.get_bwe_estimate() {
// #23: Cap BWE to prevent runaway bitrate escalation. Without this, BWE
// estimates can rise to 10+ Mbps, causing IDR bursts and PLI storms.
let effective_bwe = bwe.min(max_bitrate);
if effective_bwe != bwe {
tracing::debug!(
bwe,
effective_bwe,
max_bitrate,
"BWE exceeds --max-bitrate cap, clamping"
);
}
let bwe = effective_bwe;
let should_send = match last_sent_bitrate {
None => true,
Some(last) => {
let diff = if bwe > last { bwe - last } else { last - bwe };
diff * 10 > last
}
};
if should_send {
let _ = bitrate_tx.try_send(BitrateCommand::UpdateBitrate { target_bps: bwe });
last_sent_bitrate = Some(bwe);
}
if last_resolution_eval.elapsed() >= Duration::from_secs(1) {
last_resolution_eval = Instant::now();
let selected = select_resolution(current_tier.0, current_tier.1, bwe, fps);
if selected != current_tier {
current_tier = selected;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
} else if let Some(next_tier) = next_upscale_tier(current_tier, initial_tier) {
let needed = resolution_bitrate_bps(next_tier.0, next_tier.1, fps);
if bwe > needed.saturating_mul(120) / 100 {
upscale_counter = upscale_counter.saturating_add(1);
if upscale_counter >= 10 {
current_tier = next_tier;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
}
} else {
upscale_counter = 0;
}
} else {
upscale_counter = 0;
}
}
}
if connected {
while let Ok(enc_frame) = webrtc_rx.try_recv() {
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks) {
tracing::debug!("WebRTC write frame error: {e}");
}
frames_sent = frames_sent.saturating_add(1);
let gap_ms = last_send
.map(|l| l.elapsed().as_secs_f64() * 1000.0)
.unwrap_or(0.0);
// Compute capture-to-send age on the sending thread so the
// frame_age stat stays accurate when batch-drained later.
let age_ms =
Some(enc_frame.capture_time.elapsed().as_secs_f64() * 1000.0);
last_send = Some(std::time::Instant::now());
let _ = sent_gap_tx.try_send((gap_ms, age_ms));
}
} else {
while webrtc_rx.try_recv().is_ok() {}
}
match webrtc_rx.recv_timeout(timeout) {
Ok(enc_frame) => {
if wrtc.is_connected() {
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks)
{
tracing::debug!("WebRTC write frame error: {e}");
}
frames_sent = frames_sent.saturating_add(1);
let gap_ms = last_send
.map(|l| l.elapsed().as_secs_f64() * 1000.0)
.unwrap_or(0.0);
let age_ms =
Some(enc_frame.capture_time.elapsed().as_secs_f64() * 1000.0);
last_send = Some(std::time::Instant::now());
let _ = sent_gap_tx.try_send((gap_ms, age_ms));
}
}
Err(crossbeam_channel::RecvTimeoutError::Timeout) => {}
Err(crossbeam_channel::RecvTimeoutError::Disconnected) => {
tracing::info!("WebRTC channel disconnected, exiting thread");
return;
}
}
}
tracing::info!("WebRTC thread exiting");
}
const RESOLUTION_TIERS: &[(u32, u32)] = &[(2560, 1440), (1920, 1080), (1280, 720)];
fn resolution_bitrate_bps(width: u32, height: u32, fps: u32) -> u64 {
5 * u64::from(width) * u64::from(height) * u64::from(fps) / 100
}
/// Conservative startup bitrate for WebRTC mode, tier-based by total pixel count.
/// BWE estimate arrives within milliseconds of client connect and overrides this;
/// the startup value only affects the first IDR. See issue #21.
fn webrtc_startup_bitrate_bps(width: u32, height: u32) -> u64 {
let pixels = u64::from(width) * u64::from(height);
if pixels <= 1_000_000 {
1_000_000
} else if pixels <= 2_500_000 {
2_000_000
} else if pixels <= 4_500_000 {
4_000_000
} else {
8_000_000
}
}
/// Select resolution tier based on BWE estimate.
/// Returns (width, height) for the selected tier.
fn select_resolution(current_w: u32, current_h: u32, bwe_bps: u64, fps: u32) -> (u32, u32) {
let current = (current_w, current_h);
let current_bitrate = resolution_bitrate_bps(current_w, current_h, fps);
if bwe_bps >= current_bitrate.saturating_mul(60) / 100 {
return current;
}
let current_index = RESOLUTION_TIERS
.iter()
.position(|&tier| tier == current)
.unwrap_or_else(|| {
RESOLUTION_TIERS
.iter()
.position(|&(w, h)| w <= current_w && h <= current_h)
.unwrap_or(RESOLUTION_TIERS.len() - 1)
});
let next_index = (current_index + 1).min(RESOLUTION_TIERS.len() - 1);
RESOLUTION_TIERS[next_index]
}
fn next_upscale_tier(current: (u32, u32), ceiling: (u32, u32)) -> Option<(u32, u32)> {
let current_index = RESOLUTION_TIERS.iter().position(|&tier| tier == current)?;
if current_index == 0 {
return None;
}
let next = RESOLUTION_TIERS[current_index - 1];
(next.0 <= ceiling.0 && next.1 <= ceiling.1).then_some(next)
}
impl Drop for StatePortal {
// 析构时自动调用 shutdown,确保编码器被刷新、资源被释放
fn drop(&mut self) {
@@ -946,7 +694,12 @@ fn resolve_drm_device(args: &Args) -> Result<Option<PathBuf>> {
/// 用于验证 DMA-BUF 元数据映射的正确性。
#[cfg(test)]
fn build_drm_descriptor(frame: &PwDmaBufFrame) -> ffmpeg_next::ffi::AVDRMFrameDescriptor {
let mut desc: ffmpeg_next::ffi::AVDRMFrameDescriptor = unsafe { std::mem::zeroed() };
let mut desc: ffmpeg_next::ffi::AVDRMFrameDescriptor = {
// SAFETY: AVDRMFrameDescriptor is a POD struct from FFmpeg's C API with no
// pointers orDrop fields; all-zero is a valid initial state. Every field is
// explicitly overwritten in the lines below before the descriptor is used.
unsafe { std::mem::zeroed() }
};
desc.nb_objects = 1; // 单个 DMA-BUF 对象
desc.objects[0].fd = frame.fd.as_raw_fd(); // DMA-BUF 文件描述符
desc.objects[0].size = 0; // 大小设为 0(内核自动确定)
@@ -969,6 +722,9 @@ mod tests {
fn make_test_frame() -> PwDmaBufFrame {
// Create a dummy fd from stderr (always valid fd 2)
// 使用 stderr(fd 2)的副本作为虚拟文件描述符
// SAFETY: stderr (fd 2) is always-open in any process; libc::dup(2) returns
// a fresh fd we solely own. OwnedFd::from_raw_fd takes ownership and closes
// it on Drop. Test-only; the fd is never actually memory-mapped.
let fd = unsafe { OwnedFd::from_raw_fd(libc::dup(2)) };
PwDmaBufFrame {
fd,
@@ -1046,53 +802,13 @@ mod tests {
assert_eq!(result, None);
}
#[test]
fn webrtc_startup_bitrate_tiers_by_pixel_count() {
assert_eq!(webrtc_startup_bitrate_bps(1280, 720), 1_000_000);
assert_eq!(webrtc_startup_bitrate_bps(1920, 1080), 2_000_000);
assert_eq!(webrtc_startup_bitrate_bps(2560, 1440), 4_000_000);
assert_eq!(webrtc_startup_bitrate_bps(3840, 2160), 8_000_000);
}
#[test]
fn select_resolution_downscales_one_tier_below_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 59 / 100, fps),
(1280, 720)
);
}
#[test]
fn select_resolution_keeps_tier_at_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 60 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_never_goes_below_720p() {
assert_eq!(select_resolution(1280, 720, 1, 30), (1280, 720));
}
#[test]
fn next_upscale_tier_respects_initial_ceiling() {
assert_eq!(
next_upscale_tier((1280, 720), (1920, 1080)),
Some((1920, 1080))
);
assert_eq!(next_upscale_tier((1920, 1080), (1920, 1080)), None);
}
/// 测试:使用自定义偏移量和 stride 构建 DRM 描述符
#[test]
fn build_drm_descriptor_custom_offset_and_stride() {
// SAFETY: same as make_test_frame — dup of stderr (fd 2), test-only.
let test_fd = unsafe { OwnedFd::from_raw_fd(libc::dup(2)) };
let frame = PwDmaBufFrame {
fd: unsafe { OwnedFd::from_raw_fd(libc::dup(2)) },
fd: test_fd,
offset: 4096, // 4KB 对齐偏移
stride: 3840 * 4, // 4K 宽度 × 4 字节
modifier: 0x0100000000000001, // AMD modifiers
@@ -1110,89 +826,4 @@ mod tests {
}
// ── issue #8 regression ──
#[test]
fn try_send_full_channel_returns_full_not_block() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
tx.send(vec![1]).unwrap();
tx.send(vec![2]).unwrap();
assert!(matches!(
tx.try_send(vec![3]),
Err(crossbeam_channel::TrySendError::Full(_))
));
assert_eq!(rx.len(), 2);
}
#[test]
fn try_send_after_rx_dropped_returns_disconnected() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
drop(rx);
assert!(matches!(
tx.try_send(vec![1]),
Err(crossbeam_channel::TrySendError::Disconnected(_))
));
}
// given: full bounded channel
// when: rx is dropped, then try_send
// expect: Disconnected, not blocking
#[test]
fn shutdown_rx_drop_prevents_deadlock_on_full_channel() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
tx.send(vec![1]).unwrap();
tx.send(vec![2]).unwrap();
drop(rx);
assert!(matches!(
tx.try_send(vec![3]),
Err(crossbeam_channel::TrySendError::Disconnected(_))
));
}
// ── Task 7: Additional resolution tier edge cases ──
#[test]
fn select_resolution_keeps_720p_when_bwe_sufficient() {
let fps = 30;
let bitrate_720 = resolution_bitrate_bps(1280, 720, fps);
assert_eq!(
select_resolution(1280, 720, bitrate_720, fps),
(1280, 720)
);
}
#[test]
fn select_resolution_downscales_1440p_to_1080p() {
let fps = 30;
let bitrate_1440 = resolution_bitrate_bps(2560, 1440, fps);
assert_eq!(
select_resolution(2560, 1440, bitrate_1440 * 59 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_1080p_to_720p_at_very_low_bwe() {
let fps = 30;
let bitrate_1080 = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, bitrate_1080 / 10, fps),
(1280, 720)
);
}
#[test]
fn next_upscale_tier_from_720p_to_1080p() {
assert_eq!(
next_upscale_tier((1280, 720), (2560, 1440)),
Some((1920, 1080))
);
}
#[test]
fn next_upscale_tier_returns_none_at_highest() {
assert_eq!(next_upscale_tier((2560, 1440), (2560, 1440)), None);
}
}
+144
View File
@@ -0,0 +1,144 @@
pub(super) const RESOLUTION_TIERS: &[(u32, u32)] = &[(2560, 1440), (1920, 1080), (1280, 720)];
pub(super) fn resolution_bitrate_bps(width: u32, height: u32, fps: u32) -> u64 {
5 * u64::from(width) * u64::from(height) * u64::from(fps) / 100
}
/// Conservative startup bitrate for WebRTC mode, tier-based by total pixel count.
/// BWE estimate arrives within milliseconds of client connect and overrides this;
/// the startup value only affects the first IDR. See issue #21.
pub(super) fn webrtc_startup_bitrate_bps(width: u32, height: u32) -> u64 {
let pixels = u64::from(width) * u64::from(height);
if pixels <= 1_000_000 {
1_000_000
} else if pixels <= 2_500_000 {
2_000_000
} else if pixels <= 4_500_000 {
4_000_000
} else {
8_000_000
}
}
/// Select resolution tier based on BWE estimate.
/// Returns (width, height) for the selected tier.
pub(super) fn select_resolution(
current_w: u32,
current_h: u32,
bwe_bps: u64,
fps: u32,
) -> (u32, u32) {
let current = (current_w, current_h);
let current_bitrate = resolution_bitrate_bps(current_w, current_h, fps);
if bwe_bps >= current_bitrate.saturating_mul(60) / 100 {
return current;
}
let current_index = RESOLUTION_TIERS
.iter()
.position(|&tier| tier == current)
.unwrap_or_else(|| {
RESOLUTION_TIERS
.iter()
.position(|&(w, h)| w <= current_w && h <= current_h)
.unwrap_or(RESOLUTION_TIERS.len() - 1)
});
let next_index = (current_index + 1).min(RESOLUTION_TIERS.len() - 1);
RESOLUTION_TIERS[next_index]
}
pub(super) fn next_upscale_tier(current: (u32, u32), ceiling: (u32, u32)) -> Option<(u32, u32)> {
let current_index = RESOLUTION_TIERS.iter().position(|&tier| tier == current)?;
if current_index == 0 {
return None;
}
let next = RESOLUTION_TIERS[current_index - 1];
(next.0 <= ceiling.0 && next.1 <= ceiling.1).then_some(next)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn webrtc_startup_bitrate_tiers_by_pixel_count() {
assert_eq!(webrtc_startup_bitrate_bps(1280, 720), 1_000_000);
assert_eq!(webrtc_startup_bitrate_bps(1920, 1080), 2_000_000);
assert_eq!(webrtc_startup_bitrate_bps(2560, 1440), 4_000_000);
assert_eq!(webrtc_startup_bitrate_bps(3840, 2160), 8_000_000);
}
#[test]
fn select_resolution_downscales_one_tier_below_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 59 / 100, fps),
(1280, 720)
);
}
#[test]
fn select_resolution_keeps_tier_at_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 60 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_never_goes_below_720p() {
assert_eq!(select_resolution(1280, 720, 1, 30), (1280, 720));
}
#[test]
fn next_upscale_tier_respects_initial_ceiling() {
assert_eq!(
next_upscale_tier((1280, 720), (1920, 1080)),
Some((1920, 1080))
);
assert_eq!(next_upscale_tier((1920, 1080), (1920, 1080)), None);
}
#[test]
fn select_resolution_keeps_720p_when_bwe_sufficient() {
let fps = 30;
let bitrate_720 = resolution_bitrate_bps(1280, 720, fps);
assert_eq!(select_resolution(1280, 720, bitrate_720, fps), (1280, 720));
}
#[test]
fn select_resolution_downscales_1440p_to_1080p() {
let fps = 30;
let bitrate_1440 = resolution_bitrate_bps(2560, 1440, fps);
assert_eq!(
select_resolution(2560, 1440, bitrate_1440 * 59 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_1080p_to_720p_at_very_low_bwe() {
let fps = 30;
let bitrate_1080 = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, bitrate_1080 / 10, fps),
(1280, 720)
);
}
#[test]
fn next_upscale_tier_from_720p_to_1080p() {
assert_eq!(
next_upscale_tier((1280, 720), (2560, 1440)),
Some((1920, 1080))
);
}
#[test]
fn next_upscale_tier_returns_none_at_highest() {
assert_eq!(next_upscale_tier((2560, 1440), (2560, 1440)), None);
}
}
+287
View File
@@ -0,0 +1,287 @@
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
use crate::avhw::{BitrateCommand, CpuNv12Frame, EncodeOutcome, EncodedH264Frame, SwEncEncode};
use crate::webrtc::WebRtcState;
use super::bitrate::{next_upscale_tier, resolution_bitrate_bps, select_resolution};
pub(super) struct EncodeThreadTiming {
pub(super) sws_us: u64,
pub(super) encode_us: u64,
pub(super) output_bytes: usize,
}
pub(super) struct EncodeThread {
pub(super) handle: Option<std::thread::JoinHandle<()>>,
pub(super) input_tx: crossbeam_channel::Sender<CpuNv12Frame>,
pub(super) timing_rx: crossbeam_channel::Receiver<EncodeThreadTiming>,
pub(super) duplicate_count: std::sync::Arc<std::sync::atomic::AtomicU64>,
}
pub(super) struct WebrtcThread {
pub(super) handle: Option<std::thread::JoinHandle<()>>,
pub(super) sent_gap_rx: crossbeam_channel::Receiver<(f64, Option<f64>)>,
}
/// Static configuration handed to the WebRTC sender thread. Immutable for the
/// thread's lifetime; a resolution tier change rebuilds the whole pipeline
/// (and spawns a new thread) rather than mutating this.
pub(super) struct WebRtcThreadConfig {
pub(super) fps: u32,
pub(super) enc_width: u32,
pub(super) enc_height: u32,
pub(super) max_bitrate: u64,
}
/// Channel endpoints owned exclusively by the WebRTC sender thread after spawn.
/// The reverse endpoints stay with StatePortal (or the encode thread) for
/// inbound/outbound traffic.
pub(super) struct WebRtcThreadChannels {
pub(super) webrtc_rx: crossbeam_channel::Receiver<EncodedH264Frame>,
pub(super) sent_gap_tx: crossbeam_channel::Sender<(f64, Option<f64>)>,
pub(super) bitrate_tx: crossbeam_channel::Sender<BitrateCommand>,
pub(super) resolution_tx: crossbeam_channel::Sender<BitrateCommand>,
}
pub(super) fn encode_thread_loop(
mut encode: SwEncEncode,
input_rx: crossbeam_channel::Receiver<CpuNv12Frame>,
timing_tx: crossbeam_channel::Sender<EncodeThreadTiming>,
duplicate_count: std::sync::Arc<std::sync::atomic::AtomicU64>,
) {
loop {
match input_rx.recv() {
Ok(frame) => {
match encode.encode_cpu_frame(&frame) {
Ok(EncodeOutcome::Encoded) => {
let t = encode.take_timing();
let _ = timing_tx.try_send(EncodeThreadTiming {
sws_us: t.sws_us,
encode_us: t.encode_us,
output_bytes: t.output_bytes,
});
}
Ok(EncodeOutcome::SkippedDuplicate) => {
duplicate_count.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
}
Ok(_) => {
// SkippedPaused / SkippedDisconnected — no counter needed
}
Err(e) => {
tracing::error!("Encode thread error: {e}");
break;
}
}
}
Err(_) => {
tracing::info!("Encode thread input closed, flushing encoder");
if let Err(e) = encode.flush() {
tracing::error!("Encode thread flush error: {e}");
}
break;
}
}
}
tracing::info!("Encode thread exiting");
}
pub(super) fn webrtc_thread_loop(
mut wrtc: WebRtcState,
config: WebRtcThreadConfig,
channels: WebRtcThreadChannels,
paused: Arc<AtomicBool>,
) {
let WebRtcThreadConfig {
fps,
enc_width,
enc_height,
max_bitrate,
} = config;
let WebRtcThreadChannels {
webrtc_rx,
sent_gap_tx,
bitrate_tx,
resolution_tx,
} = channels;
let mut frames_sent: u64 = 0;
let mut last_send: Option<std::time::Instant> = None;
let mut last_sent_bitrate: Option<u64> = None;
let initial_tier = (enc_width, enc_height);
let mut current_tier = initial_tier;
let mut upscale_counter = 0u32;
let mut last_resolution_eval = Instant::now();
let timeout = Duration::from_millis(1);
loop {
if let Err(e) = wrtc.handle_signaling() {
tracing::error!("WebRTC signaling error: {e}");
break;
}
if let Err(e) = wrtc.poll_and_feed() {
tracing::error!("WebRTC poll error: {e}");
break;
}
if wrtc.take_force_keyframe() {
let _ = bitrate_tx.try_send(BitrateCommand::ForceKeyframe);
}
let connected = wrtc.is_connected();
let was_paused = paused.load(Ordering::Relaxed);
let now_paused = !connected;
if was_paused && !now_paused {
tracing::info!("WebRTC client connected, resuming encoding");
} else if !was_paused && now_paused {
tracing::warn!("WebRTC client disconnected, pausing encoding");
}
paused.store(now_paused, Ordering::Relaxed);
if let Some(bwe) = wrtc.get_bwe_estimate() {
// #23: Cap BWE to prevent runaway bitrate escalation. Without this, BWE
// estimates can rise to 10+ Mbps, causing IDR bursts and PLI storms.
let effective_bwe = bwe.min(max_bitrate);
if effective_bwe != bwe {
tracing::debug!(
bwe,
effective_bwe,
max_bitrate,
"BWE exceeds --max-bitrate cap, clamping"
);
}
let bwe = effective_bwe;
let should_send = match last_sent_bitrate {
None => true,
Some(last) => {
let diff = bwe.abs_diff(last);
diff * 10 > last
}
};
if should_send {
let _ = bitrate_tx.try_send(BitrateCommand::UpdateBitrate { target_bps: bwe });
last_sent_bitrate = Some(bwe);
}
if last_resolution_eval.elapsed() >= Duration::from_secs(1) {
last_resolution_eval = Instant::now();
let selected = select_resolution(current_tier.0, current_tier.1, bwe, fps);
if selected != current_tier {
current_tier = selected;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
} else if let Some(next_tier) = next_upscale_tier(current_tier, initial_tier) {
let needed = resolution_bitrate_bps(next_tier.0, next_tier.1, fps);
if bwe > needed.saturating_mul(120) / 100 {
upscale_counter = upscale_counter.saturating_add(1);
if upscale_counter >= 10 {
current_tier = next_tier;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
}
} else {
upscale_counter = 0;
}
} else {
upscale_counter = 0;
}
}
}
if connected {
while let Ok(enc_frame) = webrtc_rx.try_recv() {
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks) {
tracing::debug!("WebRTC write frame error: {e}");
}
frames_sent = frames_sent.saturating_add(1);
let gap_ms = last_send
.map(|l| l.elapsed().as_secs_f64() * 1000.0)
.unwrap_or(0.0);
// Compute capture-to-send age on the sending thread so the
// frame_age stat stays accurate when batch-drained later.
let age_ms = Some(enc_frame.capture_time.elapsed().as_secs_f64() * 1000.0);
last_send = Some(std::time::Instant::now());
let _ = sent_gap_tx.try_send((gap_ms, age_ms));
}
} else {
while webrtc_rx.try_recv().is_ok() {}
}
match webrtc_rx.recv_timeout(timeout) {
Ok(enc_frame) => {
if wrtc.is_connected() {
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks) {
tracing::debug!("WebRTC write frame error: {e}");
}
frames_sent = frames_sent.saturating_add(1);
let gap_ms = last_send
.map(|l| l.elapsed().as_secs_f64() * 1000.0)
.unwrap_or(0.0);
let age_ms = Some(enc_frame.capture_time.elapsed().as_secs_f64() * 1000.0);
last_send = Some(std::time::Instant::now());
let _ = sent_gap_tx.try_send((gap_ms, age_ms));
}
}
Err(crossbeam_channel::RecvTimeoutError::Timeout) => {}
Err(crossbeam_channel::RecvTimeoutError::Disconnected) => {
tracing::info!("WebRTC channel disconnected, exiting thread");
return;
}
}
}
tracing::info!("WebRTC thread exiting");
}
#[cfg(test)]
mod tests {
#[test]
fn try_send_full_channel_returns_full_not_block() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
tx.send(vec![1]).unwrap();
tx.send(vec![2]).unwrap();
assert!(matches!(
tx.try_send(vec![3]),
Err(crossbeam_channel::TrySendError::Full(_))
));
assert_eq!(rx.len(), 2);
}
#[test]
fn try_send_after_rx_dropped_returns_disconnected() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
drop(rx);
assert!(matches!(
tx.try_send(vec![1]),
Err(crossbeam_channel::TrySendError::Disconnected(_))
));
}
// given: full bounded channel
// when: rx is dropped, then try_send
// expect: Disconnected, not blocking
#[test]
fn shutdown_rx_drop_prevents_deadlock_on_full_channel() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
tx.send(vec![1]).unwrap();
tx.send(vec![2]).unwrap();
drop(rx);
assert!(matches!(
tx.try_send(vec![3]),
Err(crossbeam_channel::TrySendError::Disconnected(_))
));
}
}
+49 -23
View File
@@ -38,7 +38,6 @@ pub struct PipelineStats {
encoded_frames: u64,
sent_frames: u64,
pipewire_dropped: u64,
over_budget_count: u64,
/// Count of frames dropped by encode thread due to Y-plane hash dedup
/// (EncodeOutcome::SkippedDuplicate). Read from atomic counter set by
/// encode thread, computed as delta since previous snapshot.
@@ -73,6 +72,12 @@ pub struct PipelineStats {
window_start: Instant,
}
impl Default for PipelineStats {
fn default() -> Self {
Self::new()
}
}
impl PipelineStats {
pub fn new() -> Self {
Self {
@@ -80,7 +85,6 @@ impl PipelineStats {
encoded_frames: 0,
sent_frames: 0,
pipewire_dropped: 0,
over_budget_count: 0,
duplicate_frames_skipped: 0,
prev_duplicate_frames_skipped: 0,
capture_queue_depth: 0,
@@ -197,7 +201,8 @@ impl PipelineStats {
/// Update duplicate frames skipped counter (absolute value from atomic).
/// Computes delta from previous value, like set_pipewire_dropped.
pub fn set_duplicate_frames_skipped(&mut self, total_skipped: u64) {
self.duplicate_frames_skipped = total_skipped.saturating_sub(self.prev_duplicate_frames_skipped);
self.duplicate_frames_skipped =
total_skipped.saturating_sub(self.prev_duplicate_frames_skipped);
self.prev_duplicate_frames_skipped = total_skipped;
}
@@ -207,11 +212,6 @@ impl PipelineStats {
self.encoded_queue_depth = encoded;
}
/// Record that a frame exceeded its time budget.
pub fn record_over_budget(&mut self) {
self.over_budget_count += 1;
}
/// Returns true if at least 1 second has elapsed since the last snapshot
/// (or since creation). If true, call `snapshot_and_reset` to get the stats.
pub fn should_snapshot(&self) -> bool {
@@ -230,7 +230,6 @@ impl PipelineStats {
encoded_frames: self.encoded_frames,
sent_frames: self.sent_frames,
pipewire_dropped: self.pipewire_dropped,
over_budget_count: self.over_budget_count,
duplicate_frames_skipped: self.duplicate_frames_skipped,
capture_queue_depth: self.capture_queue_depth,
encoded_queue_depth: self.encoded_queue_depth,
@@ -269,7 +268,6 @@ impl PipelineStats {
self.encoded_frames = 0;
self.sent_frames = 0;
self.pipewire_dropped = 0;
self.over_budget_count = 0;
self.duplicate_frames_skipped = 0;
self.capture_queue_depth = 0;
self.encoded_queue_depth = 0;
@@ -304,7 +302,6 @@ pub struct StatsSnapshot {
pub encoded_frames: u64,
pub sent_frames: u64,
pub pipewire_dropped: u64,
pub over_budget_count: u64,
pub duplicate_frames_skipped: u64,
// Queue depths
pub capture_queue_depth: usize,
@@ -346,43 +343,72 @@ pub struct StatsSnapshot {
impl std::fmt::Display for StatsSnapshot {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
// Layout note: each line answers one operational question.
// - line 1: throughput (fps + frame counts + window length)
// - line 2: drops (PipeWire backlog, encoder over-budget, frame-hash dedup)
// - line 3: queue back-pressure (capture + encoded)
// - line 4-6: gap timing (avg + p95 + max) — answers "is cadence stable?"
// - line 7: capture-to-send age (avg + p95 + max) — answers "how stale?"
// - line 8: per-stage encode timing (avg + p95) — answers "where is latency?"
// - line 9: output bandwidth (bytes/sec + per-frame p95/max)
//
// The avg counterparts were computed but never displayed before Oracle
// audit 2026-06-28; they pair with the existing p95/max to surface both
// central tendency and tail behaviour in the same glance.
write!(
f,
"capture_fps={:.1} encoded_fps={:.1} sent_fps={:.1} \
pw_dropped={} over_budget={} duplicate_frames_skipped={} \
cap_q={} enc_q={} \
cap_gap_p95={:.1}ms cap_gap_max={:.1}ms \
enc_gap_p95={:.1}ms enc_gap_max={:.1}ms \
sent_gap_p95={:.1}ms sent_gap_max={:.1}ms \
frame_age_p95={:.1}ms frame_age_max={:.1}ms \
send_wait_p95={:.1}ms \
import_p95={:.1}ms scale_p95={:.1}ms transfer_p95={:.1}ms \
sws_p95={:.1}ms encode_p95={:.1}ms total_p95={:.1}ms \
output_bps={:.0} frame_bytes_max={}",
"elapsed={:.1}s capture_fps={:.1} encoded_fps={:.1} sent_fps={:.1} \
capture_frames={} encoded_frames={} sent_frames={} \
pw_dropped={} duplicate_frames_skipped={} \
cap_q={} enc_q={} \
cap_gap_avg={:.1}ms cap_gap_p95={:.1}ms cap_gap_max={:.1}ms \
enc_gap_avg={:.1}ms enc_gap_p95={:.1}ms enc_gap_max={:.1}ms \
sent_gap_avg={:.1}ms sent_gap_p95={:.1}ms sent_gap_max={:.1}ms \
frame_age_avg={:.1}ms frame_age_p95={:.1}ms frame_age_max={:.1}ms \
send_wait_p95={:.1}ms \
import_avg={:.1}ms import_p95={:.1}ms \
scale_avg={:.1}ms scale_p95={:.1}ms transfer_avg={:.1}ms transfer_p95={:.1}ms \
sws_avg={:.1}ms sws_p95={:.1}ms \
encode_avg={:.1}ms encode_p95={:.1}ms total_avg={:.1}ms total_p95={:.1}ms \
output_bps={:.0} frame_bytes_p95={} frame_bytes_max={}",
self.elapsed_secs,
self.capture_fps,
self.encoded_fps,
self.sent_fps,
self.capture_frames,
self.encoded_frames,
self.sent_frames,
self.pipewire_dropped,
self.over_budget_count,
self.duplicate_frames_skipped,
self.capture_queue_depth,
self.encoded_queue_depth,
self.capture_gap_avg_ms,
self.capture_gap_p95_ms,
self.capture_gap_max_ms,
self.encoded_gap_avg_ms,
self.encoded_gap_p95_ms,
self.encoded_gap_max_ms,
self.sent_gap_avg_ms,
self.sent_gap_p95_ms,
self.sent_gap_max_ms,
self.frame_age_avg_ms,
self.frame_age_p95_ms,
self.frame_age_max_ms,
self.send_wait_p95_ms,
self.import_avg_ms,
self.import_p95_ms,
self.scale_avg_ms,
self.scale_p95_ms,
self.transfer_avg_ms,
self.transfer_p95_ms,
self.sws_avg_ms,
self.sws_p95_ms,
self.encode_avg_ms,
self.encode_p95_ms,
self.total_avg_ms,
self.total_p95_ms,
self.output_bytes_per_sec,
self.output_frame_bytes_p95,
self.output_frame_bytes_max,
)
}
+9 -311
View File
@@ -1,8 +1,12 @@
/// Coordinate transformation module for Wayland output transforms.
///
/// Handles the 8 `wl_output` transform variants (rotation + reflection)
/// and ROI clipping for screen capture.
///
//! Coordinate transformation module for Wayland output transforms.
//!
//! Historically exposed a family of `Rect`/`screen_to_frame`/`fit_inside_bounds`
//! helpers for ROI-based capture clipping. Those were never wired into the
//! capture pipeline (we capture full frames and let FFmpeg's filter graph handle
//! any scaling/rotation); they have been removed. Only `Transform` and the
//! `transpose_if_transform_transposed` helper remain — both are actively used by
//! `state.rs` and `avhw.rs`.
/// Wayland output transform enum, matching `wl_output::Transform`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Transform {
@@ -16,68 +20,6 @@ pub enum Transform {
Flipped270,
}
/// Axis-aligned rectangle in integer coordinates.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Rect {
pub x: i32,
pub y: i32,
pub w: i32,
pub h: i32,
}
/// Returns the 2×2 basis matrix (a, b, c, d) for the given transform.
///
/// The matrix represents the affine mapping from screen coordinates to
/// frame coordinates:
///
/// ```text
/// [new_x] [a b] [x]
/// [new_y] = [c d] [y]
/// ```
pub fn transform_basis(transform: Transform) -> (i32, i32, i32, i32) {
match transform {
Transform::Normal => (1, 0, 0, 1),
Transform::Normal90 => (0, 1, -1, 0),
Transform::Normal180 => (-1, 0, 0, -1),
Transform::Normal270 => (0, -1, 1, 0),
Transform::Flipped => (-1, 0, 0, 1),
Transform::Flipped90 => (0, 1, 1, 0),
Transform::Flipped180 => (1, 0, 0, -1),
Transform::Flipped270 => (0, -1, -1, 0),
}
}
/// Transform a rectangle from screen space to frame space.
///
/// Applies the 2×2 basis matrix and computes offsets so the result
/// fits within the frame dimensions `(frame_w, frame_h)`.
///
/// ```text
/// new_x = a * x + b * y + offset_x
/// new_y = c * x + d * y + offset_y
/// ```
pub fn screen_to_frame(transform: Transform, rect: Rect, frame_w: i32, frame_h: i32) -> Rect {
let (a, b, c, d) = transform_basis(transform);
// Compute the offset so that the transformed origin maps correctly.
// For transforms with negative components, we need to shift by the
// frame dimension to keep coordinates in [0, frame_w) × [0, frame_h).
let offset_x = if a + b < 0 { frame_w } else { 0 };
let offset_y = if c + d < 0 { frame_h } else { 0 };
let new_x = a * rect.x + b * rect.y + offset_x;
let new_y = c * rect.x + d * rect.y + offset_y;
let new_w = a * rect.w + b * rect.h;
let new_h = c * rect.w + d * rect.h;
Rect {
x: new_x,
y: new_y,
w: new_w.abs(),
h: new_h.abs(),
}
}
/// Swap width and height for 90° or 270° rotations.
///
/// After a quarter-turn rotation the output dimensions are transposed
@@ -93,140 +35,10 @@ pub fn transpose_if_transform_transposed(transform: Transform, w: i32, h: i32) -
}
}
/// Clip a rectangle so it stays inside `(0, 0) .. (bounds_w, bounds_h)`.
///
/// The resulting rectangle has non-negative origin and its extent does
/// not exceed the bounds.
pub fn fit_inside_bounds(rect: Rect, bounds_w: i32, bounds_h: i32) -> Rect {
let x = rect.x.clamp(0, bounds_w);
let y = rect.y.clamp(0, bounds_h);
let right = (rect.x + rect.w).min(bounds_w);
let bottom = (rect.y + rect.h).min(bounds_h);
let w = (right - x).max(0);
let h = (bottom - y).max(0);
Rect { x, y, w, h }
}
#[cfg(test)]
mod tests {
use super::*;
// ── transform_basis ───────────────────────────────────────────
#[test]
fn basis_normal_is_identity() {
assert_eq!(transform_basis(Transform::Normal), (1, 0, 0, 1));
}
#[test]
fn basis_90_cw_rotation() {
assert_eq!(transform_basis(Transform::Normal90), (0, 1, -1, 0));
}
#[test]
fn basis_180_rotation() {
assert_eq!(transform_basis(Transform::Normal180), (-1, 0, 0, -1));
}
#[test]
fn basis_270_cw_rotation() {
assert_eq!(transform_basis(Transform::Normal270), (0, -1, 1, 0));
}
#[test]
fn basis_flipped_horizontal() {
assert_eq!(transform_basis(Transform::Flipped), (-1, 0, 0, 1));
}
#[test]
fn basis_flipped_90() {
assert_eq!(transform_basis(Transform::Flipped90), (0, 1, 1, 0));
}
#[test]
fn basis_flipped_180() {
assert_eq!(transform_basis(Transform::Flipped180), (1, 0, 0, -1));
}
#[test]
fn basis_flipped_270() {
assert_eq!(transform_basis(Transform::Flipped270), (0, -1, -1, 0));
}
// ── screen_to_frame ───────────────────────────────────────────
#[test]
fn screen_to_frame_identity_unchanged() {
let rect = Rect {
x: 10,
y: 20,
w: 100,
h: 50,
};
let result = screen_to_frame(Transform::Normal, rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 10,
y: 20,
w: 100,
h: 50
}
);
}
#[test]
fn screen_to_frame_90_rotates_origin() {
// 90° CW: top-left (0,0) in screen should map to bottom-left in frame
let rect = Rect {
x: 0,
y: 0,
w: 100,
h: 50,
};
let result = screen_to_frame(Transform::Normal90, rect, 1080, 1920);
// a=0,b=1,c=-1,d=0 => offset_x=0, offset_y=1920 (c+d=-1<0)
// new_x = 0*0 + 1*0 + 0 = 0
// new_y = -1*0 + 0*0 + 1920 = 1920
assert_eq!(result.x, 0);
assert_eq!(result.y, 1920);
// w' = 0*100 + 1*50 = 50, h' = -1*100 + 0*50 = -100 -> abs=100
assert_eq!(result.w, 50);
assert_eq!(result.h, 100);
}
#[test]
fn screen_to_frame_180_rotates() {
let rect = Rect {
x: 100,
y: 200,
w: 300,
h: 400,
};
let result = screen_to_frame(Transform::Normal180, rect, 1920, 1080);
// a=-1,b=0,c=0,d=-1, offset_x=1920, offset_y=1080
assert_eq!(result.x, -100 + 1920);
assert_eq!(result.y, -200 + 1080);
assert_eq!(result.w, 300);
assert_eq!(result.h, 400);
}
#[test]
fn screen_to_frame_flipped_horizontal() {
let rect = Rect {
x: 50,
y: 30,
w: 200,
h: 100,
};
let result = screen_to_frame(Transform::Flipped, rect, 1920, 1080);
// a=-1,b=0,c=0,d=1, offset_x=1920, offset_y=0
assert_eq!(result.x, -50 + 1920);
assert_eq!(result.y, 30);
assert_eq!(result.w, 200);
assert_eq!(result.h, 100);
}
// ── transpose_if_transform_transposed ─────────────────────────
#[test]
@@ -292,118 +104,4 @@ mod tests {
(1080, 1920)
);
}
// ── fit_inside_bounds ─────────────────────────────────────────
#[test]
fn fit_inside_already_fits() {
let rect = Rect {
x: 10,
y: 20,
w: 100,
h: 50,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(result, rect);
}
#[test]
fn fit_inside_clips_right_and_bottom() {
let rect = Rect {
x: 1800,
y: 1000,
w: 200,
h: 200,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 1800,
y: 1000,
w: 120,
h: 80
}
);
}
#[test]
fn fit_inside_clips_negative_origin() {
let rect = Rect {
x: -50,
y: -30,
w: 200,
h: 200,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 0,
y: 0,
w: 150,
h: 170
}
);
}
#[test]
fn fit_inside_completely_out_of_bounds() {
let rect = Rect {
x: 2000,
y: 2000,
w: 100,
h: 100,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 1920,
y: 1080,
w: 0,
h: 0
}
);
}
#[test]
fn fit_inside_zero_size_rect() {
let rect = Rect {
x: 100,
y: 100,
w: 0,
h: 0,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 100,
y: 100,
w: 0,
h: 0
}
);
}
#[test]
fn fit_inside_zero_bounds() {
let rect = Rect {
x: 0,
y: 0,
w: 100,
h: 100,
};
let result = fit_inside_bounds(rect, 0, 0);
assert_eq!(
result,
Rect {
x: 0,
y: 0,
w: 0,
h: 0
}
);
}
}
+3 -173
View File
@@ -17,178 +17,8 @@ use str0m::{Candidate, Event, IceConnectionState, Input, Output, Rtc, RtcConfig}
/// bursts. See issue #23.
const FORCED_KEYFRAME_MIN_INTERVAL: Duration = Duration::from_secs(1);
// ── 嵌入式 HTML 测试页面 ──────────────────────────────────────────────────
const HTML_PAGE: &str = r#"<!DOCTYPE html>
<html>
<head><title>wl-webrtc P0</title>
<style>body{background:#000;color:#fff;font-family:monospace;display:flex;flex-direction:column;align-items:center;justify-content:center;height:100vh;margin:0}
video{max-width:90vw;max-height:80vh;border:1px solid #333}
#status{margin:12px;font-size:14px;color:#aaa}
#debug{position:fixed;bottom:8px;left:8px;font-size:11px;color:#666;max-width:90vw;white-space:pre-wrap}
#stats-panel{position:fixed;top:8px;right:8px;background:rgba(0,0,0,0.7);color:#0f0;font:11px monospace;padding:6px 10px;border-radius:4px;z-index:100;pointer-events:none;max-width:90vw;white-space:pre;line-height:1.5}
</style></head>
<body>
<div id="status">Connecting...</div>
<video id="video" autoplay playsinline muted></video>
<pre id="debug"></pre>
<div id="stats-panel"></div>
<script>
const status = document.getElementById('status');
const video = document.getElementById('video');
const debug = document.getElementById('debug');
let pc = null;
const log = msg => { debug.textContent += msg + '\n'; console.log(msg); };
function preferH264(sdp) {
const lines = sdp.split('\r\n');
const h264Pts = lines
.filter(line => line.startsWith('a=rtpmap:') && line.toUpperCase().includes('H264/90000'))
.map(line => line.match(/^a=rtpmap:(\d+)/)?.[1])
.filter(Boolean);
if (h264Pts.length === 0) return sdp;
return lines.map(line => {
if (!line.startsWith('m=video ')) return line;
const parts = line.split(' ');
const header = parts.slice(0, 3);
const pts = parts.slice(3);
const preferred = h264Pts.filter(pt => pts.includes(pt));
const rest = pts.filter(pt => !preferred.includes(pt));
return [...header, ...preferred, ...rest].join(' ');
}).join('\r\n');
}
function installStatsLogger(peer) {
const panel = document.getElementById('stats-panel');
let prev = null;
const intervalSecs = 1;
setInterval(() => {
if (peer !== pc) return;
peer.getStats().then(stats => {
let rtp = null, rtt = null, codecStr = '';
let freezeCount = null, totalFreezesDuration = null;
stats.forEach(report => {
if (report.type === 'inbound-rtp' && report.kind === 'video') rtp = report;
if (report.type === 'codec' && report.mimeType && report.mimeType.includes('H264'))
codecStr = report.mimeType + ' ' + (report.payloadType || '');
// candidate-pair: feature-detect 'selected' property
if (report.type === 'candidate-pair') {
const isSel = ('selected' in report) ? report.selected : report.state === 'succeeded';
if (isSel && typeof report.currentRoundTripTime === 'number') rtt = report.currentRoundTripTime;
}
});
// Freeze stats (feature-detect)
if (rtp && typeof rtp.freezeCount !== 'undefined') {
freezeCount = rtp.freezeCount;
totalFreezesDuration = rtp.totalFreezesDuration;
}
if (!rtp) return;
const cur = {
framesDecoded: rtp.framesDecoded || 0,
framesDropped: rtp.framesDropped || 0,
framesPerSecond: rtp.framesPerSecond || 0,
packetsLost: rtp.packetsLost || 0,
jitter: rtp.jitter || 0,
bytesReceived: rtp.bytesReceived || 0,
totalDecodeTime: rtp.totalDecodeTime || 0,
jitterBufferDelay: rtp.jitterBufferDelay || 0,
jitterBufferEmittedCount: rtp.jitterBufferEmittedCount || 0,
freezeCount: freezeCount,
totalFreezesDuration: totalFreezesDuration,
rtt: rtt,
};
// Raw log to debug element (backward compat)
log('RTP-in: decoded=' + cur.framesDecoded + ' lost=' + cur.packetsLost +
' bytes=' + cur.bytesReceived + ' fps=' + cur.framesPerSecond +
(codecStr ? ' codec=' + codecStr : ''));
if (!prev) { prev = cur; return; }
// Compute deltas
const dFrames = cur.framesDecoded - prev.framesDecoded;
const dDropped = cur.framesDropped - prev.framesDropped;
const dLost = cur.packetsLost - prev.packetsLost;
const dBytes = cur.bytesReceived - prev.bytesReceived;
const dDecodeTime = cur.totalDecodeTime - prev.totalDecodeTime;
const dJitterBufDelay = cur.jitterBufferDelay - prev.jitterBufferDelay;
const dJitterBufCount = cur.jitterBufferEmittedCount - prev.jitterBufferEmittedCount;
const kbps = Math.round(dBytes * 8 / intervalSecs / 1000);
const decodeMs = dFrames > 0 ? (dDecodeTime / dFrames * 1000).toFixed(1) : '—';
const jitterBufMs = dJitterBufCount > 0 ? (dJitterBufDelay / dJitterBufCount * 1000).toFixed(1) : '—';
const jitterMs = (cur.jitter * 1000).toFixed(1);
const rttMs = cur.rtt !== null ? (cur.rtt * 1000).toFixed(1) : null;
let line = 'FPS:' + cur.framesPerSecond +
' Decoded:' + cur.framesDecoded + '(+' + dFrames + ')' +
' Dropped:' + cur.framesDropped + (dDropped > 0 ? '(+' + dDropped + ')' : '') +
' Lost:' + dLost +
' Jitter:' + jitterMs + 'ms' +
(rttMs !== null ? ' RTT:' + rttMs + 'ms' : '') +
' Decode:' + decodeMs + 'ms' +
' JBuf:' + jitterBufMs + 'ms';
if (freezeCount !== null) {
const dFreeze = cur.freezeCount - (prev.freezeCount || 0);
if (cur.freezeCount > 0 || dFreeze > 0)
line += ' Freeze:' + cur.freezeCount + '(+' + dFreeze + ')';
}
line += ' ' + kbps + 'kbps';
panel.textContent = line;
prev = cur;
}).catch(() => {});
}, intervalSecs * 1000);
}
function connect() {
if (pc) pc.close();
pc = new RTCPeerConnection();
const peer = pc;
peer.ontrack = e => {
log('ontrack: streams=' + e.streams.length + ' kind=' + e.track.kind);
video.srcObject = e.streams[0];
status.textContent = 'Track received';
};
peer.oniceconnectionstatechange = () => {
log('ICE: ' + peer.iceConnectionState);
status.textContent = 'ICE: ' + peer.iceConnectionState;
};
peer.addTransceiver('video', { direction: 'recvonly' });
installStatsLogger(peer);
peer.createOffer().then(offer => {
offer.sdp = preferH264(offer.sdp);
return peer.setLocalDescription(offer);
})
.then(() => new Promise(resolve => {
if (peer.iceGatheringState === 'complete') resolve();
else peer.onicegatheringstatechange = () => { if (peer.iceGatheringState === 'complete') resolve(); };
}))
.then(() => fetch('/sdp', { method: 'POST', body: JSON.stringify(peer.localDescription) }))
.then(r => { if (!r.ok) throw new Error('SDP exchange failed: ' + r.status); return r.json(); })
.then(answer => { if (answer.error) throw new Error(answer.error); return peer.setRemoteDescription(answer); })
.then(() => log('SDP answer set'))
.catch(e => {
status.textContent = 'Error: ' + e.message;
log('ERROR: ' + e.message + ' — retrying in 2s...');
console.error(e);
setTimeout(connect, 2000);
});
}
connect();
</script>
</body></html>"#;
mod html_page;
use html_page::HTML_PAGE;
// ── WebRTC 状态 ───────────────────────────────────────────────────────────
@@ -536,7 +366,7 @@ impl WebRtcInner {
let now = Instant::now();
let should_honor = self
.last_forced_keyframe_at
.map_or(true, |last| now.duration_since(last) >= FORCED_KEYFRAME_MIN_INTERVAL);
.is_none_or(|last| now.duration_since(last) >= FORCED_KEYFRAME_MIN_INTERVAL);
if should_honor {
self.last_forced_keyframe_at = Some(now);
self.need_keyframe = true;
+170
View File
@@ -0,0 +1,170 @@
pub(super) const HTML_PAGE: &str = r#"<!DOCTYPE html>
<html>
<head><title>wl-webrtc P0</title>
<style>body{background:#000;color:#fff;font-family:monospace;display:flex;flex-direction:column;align-items:center;justify-content:center;height:100vh;margin:0}
video{max-width:90vw;max-height:80vh;border:1px solid #333}
#status{margin:12px;font-size:14px;color:#aaa}
#debug{position:fixed;bottom:8px;left:8px;font-size:11px;color:#666;max-width:90vw;white-space:pre-wrap}
#stats-panel{position:fixed;top:8px;right:8px;background:rgba(0,0,0,0.7);color:#0f0;font:11px monospace;padding:6px 10px;border-radius:4px;z-index:100;pointer-events:none;max-width:90vw;white-space:pre;line-height:1.5}
</style></head>
<body>
<div id="status">Connecting...</div>
<video id="video" autoplay playsinline muted></video>
<pre id="debug"></pre>
<div id="stats-panel"></div>
<script>
const status = document.getElementById('status');
const video = document.getElementById('video');
const debug = document.getElementById('debug');
let pc = null;
const log = msg => { debug.textContent += msg + '\n'; console.log(msg); };
function preferH264(sdp) {
const lines = sdp.split('\r\n');
const h264Pts = lines
.filter(line => line.startsWith('a=rtpmap:') && line.toUpperCase().includes('H264/90000'))
.map(line => line.match(/^a=rtpmap:(\d+)/)?.[1])
.filter(Boolean);
if (h264Pts.length === 0) return sdp;
return lines.map(line => {
if (!line.startsWith('m=video ')) return line;
const parts = line.split(' ');
const header = parts.slice(0, 3);
const pts = parts.slice(3);
const preferred = h264Pts.filter(pt => pts.includes(pt));
const rest = pts.filter(pt => !preferred.includes(pt));
return [...header, ...preferred, ...rest].join(' ');
}).join('\r\n');
}
function installStatsLogger(peer) {
const panel = document.getElementById('stats-panel');
let prev = null;
const intervalSecs = 1;
setInterval(() => {
if (peer !== pc) return;
peer.getStats().then(stats => {
let rtp = null, rtt = null, codecStr = '';
let freezeCount = null, totalFreezesDuration = null;
stats.forEach(report => {
if (report.type === 'inbound-rtp' && report.kind === 'video') rtp = report;
if (report.type === 'codec' && report.mimeType && report.mimeType.includes('H264'))
codecStr = report.mimeType + ' ' + (report.payloadType || '');
// candidate-pair: feature-detect 'selected' property
if (report.type === 'candidate-pair') {
const isSel = ('selected' in report) ? report.selected : report.state === 'succeeded';
if (isSel && typeof report.currentRoundTripTime === 'number') rtt = report.currentRoundTripTime;
}
});
// Freeze stats (feature-detect)
if (rtp && typeof rtp.freezeCount !== 'undefined') {
freezeCount = rtp.freezeCount;
totalFreezesDuration = rtp.totalFreezesDuration;
}
if (!rtp) return;
const cur = {
framesDecoded: rtp.framesDecoded || 0,
framesDropped: rtp.framesDropped || 0,
framesPerSecond: rtp.framesPerSecond || 0,
packetsLost: rtp.packetsLost || 0,
jitter: rtp.jitter || 0,
bytesReceived: rtp.bytesReceived || 0,
totalDecodeTime: rtp.totalDecodeTime || 0,
jitterBufferDelay: rtp.jitterBufferDelay || 0,
jitterBufferEmittedCount: rtp.jitterBufferEmittedCount || 0,
freezeCount: freezeCount,
totalFreezesDuration: totalFreezesDuration,
rtt: rtt,
};
// Raw log to debug element (backward compat)
log('RTP-in: decoded=' + cur.framesDecoded + ' lost=' + cur.packetsLost +
' bytes=' + cur.bytesReceived + ' fps=' + cur.framesPerSecond +
(codecStr ? ' codec=' + codecStr : ''));
if (!prev) { prev = cur; return; }
// Compute deltas
const dFrames = cur.framesDecoded - prev.framesDecoded;
const dDropped = cur.framesDropped - prev.framesDropped;
const dLost = cur.packetsLost - prev.packetsLost;
const dBytes = cur.bytesReceived - prev.bytesReceived;
const dDecodeTime = cur.totalDecodeTime - prev.totalDecodeTime;
const dJitterBufDelay = cur.jitterBufferDelay - prev.jitterBufferDelay;
const dJitterBufCount = cur.jitterBufferEmittedCount - prev.jitterBufferEmittedCount;
const kbps = Math.round(dBytes * 8 / intervalSecs / 1000);
const decodeMs = dFrames > 0 ? (dDecodeTime / dFrames * 1000).toFixed(1) : '—';
const jitterBufMs = dJitterBufCount > 0 ? (dJitterBufDelay / dJitterBufCount * 1000).toFixed(1) : '—';
const jitterMs = (cur.jitter * 1000).toFixed(1);
const rttMs = cur.rtt !== null ? (cur.rtt * 1000).toFixed(1) : null;
let line = 'FPS:' + cur.framesPerSecond +
' Decoded:' + cur.framesDecoded + '(+' + dFrames + ')' +
' Dropped:' + cur.framesDropped + (dDropped > 0 ? '(+' + dDropped + ')' : '') +
' Lost:' + dLost +
' Jitter:' + jitterMs + 'ms' +
(rttMs !== null ? ' RTT:' + rttMs + 'ms' : '') +
' Decode:' + decodeMs + 'ms' +
' JBuf:' + jitterBufMs + 'ms';
if (freezeCount !== null) {
const dFreeze = cur.freezeCount - (prev.freezeCount || 0);
if (cur.freezeCount > 0 || dFreeze > 0)
line += ' Freeze:' + cur.freezeCount + '(+' + dFreeze + ')';
}
line += ' ' + kbps + 'kbps';
panel.textContent = line;
prev = cur;
}).catch(() => {});
}, intervalSecs * 1000);
}
function connect() {
if (pc) pc.close();
pc = new RTCPeerConnection();
const peer = pc;
peer.ontrack = e => {
log('ontrack: streams=' + e.streams.length + ' kind=' + e.track.kind);
video.srcObject = e.streams[0];
status.textContent = 'Track received';
};
peer.oniceconnectionstatechange = () => {
log('ICE: ' + peer.iceConnectionState);
status.textContent = 'ICE: ' + peer.iceConnectionState;
};
peer.addTransceiver('video', { direction: 'recvonly' });
installStatsLogger(peer);
peer.createOffer().then(offer => {
offer.sdp = preferH264(offer.sdp);
return peer.setLocalDescription(offer);
})
.then(() => new Promise(resolve => {
if (peer.iceGatheringState === 'complete') resolve();
else peer.onicegatheringstatechange = () => { if (peer.iceGatheringState === 'complete') resolve(); };
}))
.then(() => fetch('/sdp', { method: 'POST', body: JSON.stringify(peer.localDescription) }))
.then(r => { if (!r.ok) throw new Error('SDP exchange failed: ' + r.status); return r.json(); })
.then(answer => { if (answer.error) throw new Error(answer.error); return peer.setRemoteDescription(answer); })
.then(() => log('SDP answer set'))
.catch(e => {
status.textContent = 'Error: ' + e.message;
log('ERROR: ' + e.message + ' — retrying in 2s...');
console.error(e);
setTimeout(connect, 2000);
});
}
connect();
</script>
</body></html>"#;