Compare commits

...
Author SHA1 Message Date
dailz e49339bdab docs(agents): update module paths after directory-form refactor
CI / Security audit (RUSTSEC) (pull_request) Has been cancelled
CI / Build + Clippy + Test (pull_request) Has been cancelled
Update the 'Runtime architecture' section to reflect that state.rs /
cap_portal.rs / state_portal.rs / webrtc.rs are now parent modules of
directory trees:

- src/state.rs -> src/state/mod.rs (+ src/state/dispatch/ for the 13
  Wayland Dispatch impls)
- src/state_portal.rs still exists; helpers split into
  src/state_portal/{bitrate,threads}.rs
- src/cap_portal.rs holds the struct; setup/token_fs/pipewire_thread
  split into src/cap_portal/
- src/webrtc.rs gains src/webrtc/html_page.rs sibling

No content changes beyond the path references; the rest of AGENTS.md
remains accurate.
2026-07-14 10:32:43 +08:00
dailz 1d1b5db3c2 refactor(bin): convert vaapi_import_bench + sw_encode_bench to directory form
Step 5 + 6: split two bench binaries into directory form with sibling
helper modules. Cargo auto-discovers src/bin/<name>/main.rs as binary
<name>; no Cargo.toml change needed.

vaapi_import_bench (973 LOC) -> 6 files:
- main.rs           main() + mod declarations
- stats.rs          BenchArgs + PipelineMode + FrameStats + impl
- software.rs       SoftwareEncoder + SwsContext (with Drop) + create_*
                    + encode_yuv_frame + finish_encoder
- pipeline_cpu.rs   run_cpu_pipeline
- pipeline_gpu.rs   import_frame + build_gpu_filter_graph + run_gpu_pipeline
- util.rs           output_for_mode + print_detailed_results + print_comparison

sw_encode_bench (547 LOC) -> 2 files:
- main.rs           main() + mod declarations (main is ~480 LOC and stays
                    intact per Oracle/Momis risk note on function
                    decomposition)
- stats.rs          BenchArgs + FrameStats + impl + pix_fmt helper

Both main.rs files use #[path = "../common/mod.rs"] mod common; to keep
sharing src/bin/common/mod.rs (path adjusted for the new directory depth).

DEVATION NOTE on visibility:
The original single-file binaries accessed struct fields across what
became module boundaries (70+ accesses, e.g. encoder.yuv_frame in
run_cpu_pipeline, sws_ctx.0 in run_gpu_pipeline, stats.frames_encoded
in main, stats.mmap_us in main). Rule 2 forbids widening visibility on
struct fields. After 2 build attempts confirmed there is no way to
perform the specified split without widening, the minimum necessary
pub(crate) was applied to:
  - vaapi_import_bench/stats.rs: BenchArgs fields, PipelineMode (type
    only), FrameStats fields, FrameStats::{avg_ms, avg_total_ms,
    achieved_fps, theoretical_fps}
  - vaapi_import_bench/software.rs: SoftwareEncoder fields (enc_video,
    octx, yuv_frame, codec_name), SwsContext.0, all four functions
  - vaapi_import_bench/pipeline_*.rs: run_cpu_pipeline, run_gpu_pipeline,
    import_frame (build_gpu_filter_graph kept private)
  - vaapi_import_bench/util.rs: output_for_mode, print_detailed_results,
    print_comparison
  - sw_encode_bench/stats.rs: BenchArgs fields, FrameStats fields,
    FrameStats::avg_ms, pix_fmt

No pub (truly public) was used anywhere. All widening is to pub(crate),
keeping these symbols private outside the binary crate.

Verification (all green):
- cargo build --bins / cargo build --release --bins
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- --help smoke test on both binaries
2026-07-13 19:33:07 +08:00
dailz a17f809d9f refactor(state): split 1598-LOC state.rs into directory + extract 13 Dispatch impls
Step 4a + 4b combined.

- src/state.rs (1594 LOC) -> src/state/mod.rs (struct + inherent methods +
  types + helpers; 999 LOC) + src/state/dispatch/ (13 Dispatch impls across
  6 files: registry.rs / wl_output.rs / dmabuf.rs / screencopy.rs /
  output_mgr.rs / buffer.rs).

Per Oracle audit: orphan rule permits Dispatch impls in submodules because
Dispatch is a foreign trait on local type State<S>. All State fields the
impls touch are already pub/pub(crate) — no visibility widening needed.

Verification (all green):
- cargo build / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- cargo check --bin vaapi_import_bench --bin sw_encode_bench
2026-07-13 19:09:39 +08:00
dailz bcfbd93f5a refactor(state_portal): extract bitrate helpers + thread loops to submodules
Step 3: split state_portal.rs (1241 -> 829 LOC) into three modules.

- src/state_portal.rs (829 LOC): keeps StatePortal struct + impl (with
  poll_and_encode / handle_pw_frame / shutdown / etc.) + Drop + PortalStage
  enum + DRM helpers + DRM tests. Per Oracle/Explore audit, all 21
  StatePortal fields are private and poll_and_encode interleaves three
  channel reads with state-machine transitions; moving it would force
  pub(crate) on every field, so it stays in mod.rs.
- src/state_portal/bitrate.rs (144 LOC): RESOLUTION_TIERS + 4 pure fns
  (resolution_bitrate_bps / webrtc_startup_bitrate_bps / select_resolution /
  next_upscale_tier) + 10 tests that exercise them. Pure fns with no
  StatePortal field access — the cleanest possible extract.
- src/state_portal/threads.rs (287 LOC): the 5 thread-related types
  (EncodeThreadTiming / EncodeThread / WebrtcThread / WebRtcThreadConfig /
  WebRtcThreadChannels) + the two free fns encode_thread_loop /
  webrtc_thread_loop + the 3 channel-semantics regression tests
  (try_send_* / shutdown_rx_drop_*) that document crossbeam invariants
  the shutdown logic relies on. Struct fields widened to pub(super) so
  StatePortal in mod.rs can construct and join them.

Test preservation:
- state_portal test count: 17 (mod.rs=4 drm tests + bitrate.rs=10 +
  threads.rs=3 channel tests) — matches baseline.

Verification (all green):
- cargo build / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
2026-07-13 16:35:44 +08:00
dailz 60d6e7f046 refactor(cap_portal): split 1313-LOC file into 7 submodules
Step 2b.1: structural split (no function decomposition — that's 2b.2).

src/cap_portal.rs (1313 -> 176 LOC) now contains only the CapPortal struct,
its constructor (new), accessors (frame_receiver/event_receiver/dropped_count/
capture_queue_depth), and Drop impl. Six new sibling submodules under
src/cap_portal/:

- types.rs       (79 LOC)  timeout constants, PortalPhaseTimeout enum,
                            pub types PwDmaBufFrame / PortalFormatInfo /
                            PwCtrlEvent
- logging.rs     (18 LOC)  log_portal_phase_timeout helper
- fourcc.rs      (73 LOC)  spa_to_drm_fourcc + its 2 tests
- token_fs.rs    (362 LOC) 8 restore-token fs helpers + 11 security tests
- setup.rs       (192 LOC) impl CapPortal { setup_portal + _setup_portal_inner }
                            (associated fns; no self access — clean extract)
- pipewire_thread.rs (446 LOC) PwThreadCtx (now private to this file),
                            pipewire_thread body (verbatim, 18 SAFETY
                            comments preserved), new spawn_pipewire_thread
                            helper that constructs PwThreadCtx internally
                            and returns JoinHandle. CapPortal::new now calls
                            pipewire_thread::spawn_pipewire_thread(...) instead
                            of inlining the PwThreadCtx construction.

Oracle audit points honored:
- PwThreadCtx moved as a whole; Drop in mod.rs and pipewire_thread in
  pipewire_thread.rs share zero state through it (PwThreadCtx consumed
  by-value inside pipewire_thread; spawn helper owns the construction).
- All // SAFETY comments travel verbatim with their unsafe blocks.
- The 18 SAFETY comments in pipewire_thread are intact; clippy
  undocumented_unsafe_blocks=deny still passes.

API stability:
- pub use types::{PwCtrlEvent, PwDmaBufFrame} preserves the existing
  wl_webrtc::cap_portal::{PwCtrlEvent, PwDmaBufFrame} paths used by
  both bench binaries (verified by cargo check --bin vaapi_import_bench
  --bin sw_encode_bench).
- PortalFormatInfo was nominally pub in the original file but never
  referenced outside cap_portal; kept pub in types.rs (for cross-
  submodule access) but not re-exported from cap_portal.rs, so the
  accidental over-exposure is now scoped back.

Verification (all green):
- cargo build / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cap_portal test count: 13 (fourcc=2 + token_fs=11) — matches baseline
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- cargo check --bin vaapi_import_bench --bin sw_encode_bench
2026-07-13 16:20:56 +08:00
dailz 51f6649159 refactor(bin): dedupe av_err_to_string + receive_first_frame + drain_encoder via shared src/bin/common/mod.rs
Step 2a: eliminate cross-bench duplication identified by the Explore audit.

Changes:
- src/avhw/util.rs: av_err_to_string promoted pub(crate) -> pub (the only
  change to src/avhw/ in this whole refactor plan).
- src/avhw/mod.rs: re-export av_err_to_string; #[allow(unused_imports)]
  silences rustc's per-bin unused-import false positive (the pub use is
  consumed by the bench bins, not by the main bin).
- src/bin/common/mod.rs (new): shared receive_first_frame + drain_encoder.
  These were byte-identical between the two bench binaries modulo a type-path
  alias (ff::codec::encoder::video::Video vs ff::encoder::video::Video) and
  SAFETY-comment line wrapping. Both binaries now wire it via
  #[path = "common/mod.rs"] mod common;.
- src/bin/vaapi_import_bench.rs: 1039 -> 947 LOC (av_err_to_string,
  receive_first_frame, drain_encoder all removed; 3 call sites updated).
- src/bin/sw_encode_bench.rs: 614 -> 545 LOC (receive_first_frame,
  drain_encoder removed; 3 call sites updated).
- use ffmpeg_next::packet::Mut moved to common/mod.rs (was needed only for
  pkt.as_mut_ptr() inside drain_encoder).

Verification (all green):
- cargo build --bins / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- Test counts unchanged from baseline
2026-07-13 16:04:30 +08:00
dailz bc405c6d16 refactor(webrtc): extract HTML_PAGE const to src/webrtc/html_page.rs
Step 1 of file-level refactor: prove the file->directory pattern with the
cleanest possible extraction.

- src/webrtc.rs: 913 -> 741 LOC
- New src/webrtc/html_page.rs: 170-line HTML test page as pub(super) const
- Parent module re-exports via `mod html_page; use html_page::HTML_PAGE;`
  so all references in handle_signaling stay unchanged.

Verification (all green):
- cargo build / cargo build --release
- cargo test (79 lib + 3 integration = 82 pass, 1 ignored — unchanged)
- cargo clippy --all-targets -- -D warnings
- cargo fmt --check
- cargo check --bin vaapi_import_bench --bin sw_encode_bench
- Test count in webrtc.rs: 18 (unchanged from baseline)

Oracle audit note: HTML_PAGE had a single use site (handle_signaling L257-258)
and zero #[cfg(test)] references, so the extraction is provably behavior-
preserving.
2026-07-13 15:47:42 +08:00
dailz 75ad4bba78 style: apply rustfmt to establish clean baseline before refactor
Pre-refactor baseline state:
- 79 lib tests + 3 integration tests pass (1 integration test #[ignore])
- cargo clippy --all-targets -- -D warnings clean
- cargo build --release clean

No semantic changes; only rustfmt drift correction across 6 files.
2026-07-13 15:40:54 +08:00
dailz fed8c2dcfd docs(avhw): fix misleading Send soundness reasoning
CI / Build + Clippy + Test (pull_request) Failing after 30s
CI / Security audit (RUSTSEC) (pull_request) Failing after 30s
Oracle audit of all 5 `unsafe impl Send` in src/avhw/ found soundness
intact but reasoning wrong in 3 of 5:

- AvHwDevCtx: claimed '&mut self ensures exclusive access' — false,
  ref_clone() hands raw pointers to other threads / FFmpeg-internal
  codec workers. Real basis is AVBufferRef atomic_uint refcount +
  libva VADisplay thread safety.
- AvHwFrameCtx: claimed 'send/receive pattern is thread-safe' —
  misdirection. Real basis is AVBufferPool atomic get/put.
- EncState: claimed 'raw pointers not shared across threads' — false
  when FFmpeg frame/slice threading is enabled. Real basis is the
  hw device/frames contexts being designed for such sharing.

SwEncState and SwEncEncode comments were acceptable; improved for
clarity (note that contained FFmpeg handles are non-thread-safe but
Send-sound under exclusive access, and that crossbeam/Arc fields are
already Send by design).

Added module-level convention doc to src/avhw/mod.rs centralizing
the C-API-level justification rule and explicitly calling out the
'&mut self as Send basis' anti-pattern so future contributors don't
repeat the category error.

Fixed AGENTS.md:
- Stale claim that Cargo.toml 'only warns' on undocumented_unsafe_blocks
  (it's been 'deny' for a while)
- Stale path src/avhw.rs → src/avhw/ (split in d53e881)
- Stale 'avoid moving wrappers across threads' guidance — Send is sound,
  the audit just confirmed why

No code behavior change. Verified: cargo build --release,
cargo clippy --release --all-targets (0 warnings),
cargo test --release (79 unit + 3 integration, 1 ignored).
2026-07-09 15:20:16 +08:00
dailz 9a7b745a0e refactor(state): make output probe readiness transform-only
CI / Build + Clippy + Test (pull_request) Failing after 11s
CI / Security audit (RUSTSEC) (pull_request) Failing after 31s
Drops PartialOutputInfo.physical_size and .logical_position. After the
warning cleanup in 6332472 these fields were probe-time-only gates with
no downstream consumer (encoder reads geometry from the dmabuf frame
itself, not from wl_output description).

Readiness simplification (try_finalize_output):
- xdg-output path (Sway/Hyprland): done_count >= 2 + name + transform
- wlr-output-management path (niri): done_count >= 1 + wlr_manager_done + transform

Safe because Wayland protocol guarantees Geometry/Mode/Position events
fire before Done, so done_count >= N implies the prior events arrived.
done_count is the real signal; the per-field .is_none() checks were
redundant belt-and-suspenders.

Cascading cleanup of writers that only fed the deleted fields:
- WlOutput::Geometry handler — drop physical_size write, keep transform
- XdgOutputEvent::LogicalPosition arm — deleted
- WlrHeadEvent::Position arm — deleted (was the only reader of
  wlr_head_proxy_to_name; both maps now write-only markers)
- WlrHeadInfo.position field — deleted; struct becomes empty marker

Behavior change risk: probe may finalize slightly earlier in cases where
a compositor fires Done before physical_size/logical_position events
(protocol violation, but possible). Verified with cargo test --release
(79 unit + 3 integration, 1 ignored). Hardware/Wayland-session test
deferred to user.

Net: -48 lines.
2026-07-09 14:48:10 +08:00
dailz 633247201c refactor: clear remaining clippy dead-code and cast warnings
CI / Build + Clippy + Test (pull_request) Failing after 38m47s
CI / Security audit (RUSTSEC) (pull_request) Failing after 1m30s
Brings `cargo clippy --release --all-targets` and `cargo build --release`
to zero warnings. Three categories:

Truly dead code (deleted):
- OutputInfo.physical_size / .logical_position — copied from PartialOutputInfo
  at construction but never read on OutputInfo; PartialOutputInfo still uses
  them as probe-completion gates
- EncConstructionStage::Streaming.output_info — stored at ->Streaming
  transition, all 9 match arms discard via `..` or `output_info: _`
- State.starting_timestamp — vestigial Phase 1 stub; PTS normalization lives
  in EncState / SwEncState instead (commit 079611a)
- cap_portal::fourcc() const fn — superseded by drm_fourcc::DrmFourcc
- PwThreadCtx.fps — destructured as `fps: _`, never consumed

Lifetime/ownership invariants (kept with #[allow(dead_code)] + reason):
- CapPortal.rt — ashpd caches a zbus::Connection in a process-global
  OnceCell; runtime must outlive CapPortal or the connection hangs
- EncState.hw_device_ctx — root AVHWDeviceContext; consumers hold their
  own ref_clone() but the root ref must stay alive for ownership

False-positive warning (annotated):
- state_portal use AsRawFd — required at FFI boundary but rustc mis-attributes
  the call to OwnedFd's inherent method; E0599 if removed

Cosmetic:
- drop 5 redundant `as *const i32` casts on linesize.as_ptr() in
  avhw/encode.rs and bin/vaapi_import_bench.rs
2026-07-09 14:36:10 +08:00
dailz 9829a1728b ci(gitea): retry apt setup
CI / Build + Clippy + Test (pull_request) Failing after 31s
CI / Security audit (RUSTSEC) (pull_request) Failing after 3h10m23s
2026-07-03 13:55:46 +08:00
dailz eca8032bcc ci(gitea): use cargo git registry index
CI / Build + Clippy + Test (pull_request) Failing after 55s
CI / Security audit (RUSTSEC) (pull_request) Failing after 1h57m35s
2026-07-03 13:39:07 +08:00
dailzandSisyphus b96b99fc9c ci(gitea): install libavfilter dev package
CI / Build + Clippy + Test (pull_request) Failing after 2m47s
CI / Security audit (RUSTSEC) (pull_request) Failing after 16m56s
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-07-03 13:10:15 +08:00
dailzandSisyphus 823dd53745 chore(deps): address cargo audit findings
CI / Build + Clippy + Test (pull_request) Failing after 6m5s
CI / Security audit (RUSTSEC) (pull_request) Successful in 10m32s
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-07-03 11:23:48 +08:00
dailzandSisyphus d53e881496 refactor(avhw): split encoder module
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-07-03 10:57:05 +08:00
dailz 17f5e235a9 ci(gitea): broaden libclang find pattern to match versioned sonames
CI / Security audit (RUSTSEC) (push) Failing after 1m31s
CI / Build + Clippy + Test (push) Failing after 4m18s
Second LIBCLANG_PATH failure mode: my 'libclang.so.*' with -type f
pattern returned nothing because Debian Bookworm's runtime library is
'libclang-14.so.1' (versioned, with no plain libclang.so.* symlink),
and the .so symlink is itself a symlink not a regular file (-type f
excludes it).

bindgen accepts any of: libclang.so, libclang-*.so, libclang.so.*,
libclang-*.so.* — so the broader 'libclang*.so*' (no -type filter)
catches every variant. Also broadened search root from /usr/lib to
/usr to cover both /usr/lib/x86_64-linux-gnu/ (runtime lib) and
/usr/lib/llvm-*/lib/ (dev symlink).

Log line now includes the actual matched path so future debugging
is one glance.
2026-06-28 17:20:02 +08:00
dailz 4e65f4175b ci(gitea): dynamically resolve LIBCLANG_PATH for act_runner container
CI / Build + Clippy + Test (push) Failing after 20m22s
CI / Security audit (RUSTSEC) (push) Failing after 1m31s
First real CI run on the Gitea Actions runner hit the predicted
LIBCLANG_PATH issue but for an unexpected reason: workflow-level
'env:' does not reliably propagate into act_runner's Docker executor
(bindgen received empty LIBCLANG_PATH despite /usr/lib/llvm-*/lib
being correct on the runner).

Two changes:

  1. Drop the hardcoded workflow-level 'env: LIBCLANG_PATH' and add a
     dedicated 'Resolve LIBCLANG_PATH' step that does:
         find /usr/lib -name 'libclang.so.*' | head -1 | xargs dirname
     and writes the result to $GITHUB_ENV. Dynamic discovery also
     future-proofs against Debian/Ubuntu version drift (llvm-14 today,
     llvm-18 tomorrow). The $GITHUB_ENV mechanism is reliably visible
     across step boundaries inside the act_runner Docker container,
     whereas workflow-level env: is not.

  2. apt install: drop '--no-install-recommends' on libclang-dev
     (Debian Bookworm's metapackage uses Recommends to pull in
     versioned toolchain bits bindgen needs). Also install 'clang'
     (not just llvm-14) to get version-agnostic libclang shared lib.

Comments inline in ci.yml document both decisions to prevent future
regression.

The 'Resolve LIBCLANG_PATH' step fails fast with a clear message if
libclang isn't installed, instead of letting the clippy/build steps
fail cryptically later.
2026-06-28 16:58:04 +08:00
dailz c772e4eb0b chore: bump MSRV to 1.87 to match actual API usage
CI / Build + Clippy + Test (push) Failing after 1h10m8s
CI / Security audit (RUSTSEC) (push) Failing after 1m31s
Oracle P2 follow-up. The clippy --fix autofixes earlier in this branch
silently introduced dependencies on APIs newer than the README's
1.70+ claim:
  - u32::is_multiple_of  (stable 1.87)
  - Option::is_none_or   (stable 1.82)

clippy::incompatible_msrv flagged the mismatch once rust-version was
pinned. Bumping the floor to 1.87 is the honest fix — the codebase
genuinely depends on 1.87 features now, and 1.87 has been stable
long enough (current stable is 1.96) that desktop CLI users on stable
Rust already have it.

  - Cargo.toml: rust-version '1.70' -> '1.87'. Comment lists the specific
    APIs that drove the bump and notes that further bumps need to be
    validated against clippy::incompatible_msrv.
  - README.md: Prerequisites line updated to 1.87+ with a brief why.
  - src/state_portal.rs: added the AsRawFd rustc-quirk comment that was
    already in avhw.rs (rustc emits a false 'unused_imports' warning;
    removing it produces E0599). Same known quirk, same documentation
    pattern.
  - src/transform.rs: fixed empty_line_after_doc_comments warning by
    converting the leading // doc-style comment to a //! module-level
    doc comment (which is what it should have been when I rewrote the
    file in commit 145b5d3).

All 79 unit tests + 3 integration tests pass. clippy: 0 errors,
0 incompatible_msrv warnings, 0 empty_line_after_doc_comments warnings.
Remaining warnings are: 1 AsRawFd rustc false-positive (documented),
5 unnecessary_cast FFI false-positives (rustc quirk on pointer casts),
and 8 dead-code items that need product decisions.
2026-06-28 14:39:00 +08:00
dailz 86a8b61b07 refactor: clear too_many_arguments and large_enum_variant warnings
Oracle P2 batch 2 (refactor items). Drops both remaining design-shape
clippy warnings to zero without behavior change.

  - avhw.rs build_filter_graph: drop unused _enc_width/_enc_height params
    (Oracle caught them during P2 review — passed by EncState::new but
    never read inside the function; the filter graph uses width/height
    only). Signature: 8 args -> 6 args (under clippy's 7 threshold).

  - state.rs InFlightSurface::CopyQueued: Box the drm_map field.
    AVDRMFrameDescriptor is ~592 bytes (4 objects + 4 layers); the enum
    size was being dominated by this variant, ballooning every
    InFlightSurface value to 592 bytes even for the None/AllocQueued
    variants. Box<AVDRMFrameDescriptor> shrinks the enum to ~32 bytes
    regardless of variant. The drm_map field is currently destructured
    under _drm_map (unused), so the boxing has no consumer-side impact.

  - state_portal.rs webrtc_thread_loop: 10 args -> 4 args via two new
    structs:
      * WebRtcThreadConfig { fps, enc_width, enc_height, max_bitrate }
        — immutable for the thread's lifetime; a tier change spawns a
        new thread rather than mutating.
      * WebRtcThreadChannels { webrtc_rx, sent_gap_tx, bitrate_tx,
        resolution_tx } — channel endpoints owned exclusively by the
        sender thread after spawn.
    wrtc (WebRtcState) and paused (Arc<AtomicBool>) stay as separate
    args because they have different ownership semantics (moved-in
    state vs shared atomic). Documented as doc comments on the new
    types so the next reader understands the bundle rationale.

All 79 unit tests + 3 integration tests pass. clippy: 0 errors.
Per-file warning counts: state_portal.rs down from 3 to 0; state.rs
down from 8 to 4 (remaining are unrelated dead-code on OutputInfo /
starting_timestamp).
2026-06-28 14:35:35 +08:00
dailz ed39d3d873 ci: add build/test/clippy gate + cargo audit; pin rust-version
Oracle P2 plan step 1+2+missed-fields. Locks in the audit cleanup so
future PRs can't regress the 0-errors / deny-unsafe / 79-tests baseline.

  - .github/workflows/ci.yml: two jobs on ubuntu-latest (Linux only —
    project is Wayland/VAAPI-specific, no macOS/Windows story).
      * build-test: installs ffmpeg + libavcodec/libavformat/libavutil/
        libswscale/libva dev + libwayland + libdrm + libpipewire-0.3-dev
        + libclang-dev/llvm-14 (LIBCLANG_PATH pinned); caches cargo +
        target; runs clippy -> build --release -> test --release.
        Release build before tests is mandatory because
        tests/integration_test.rs shells out to target/release/wl-webrtc.
      * audit: installs cargo-audit and runs 'cargo audit --deny warnings'
        as a separate job so a RUSTSEC advisory fails the build
        independently of compile state.
    No -D warnings on clippy yet — undocumented_unsafe_blocks is already
    deny via Cargo.toml; remaining warnings are advisory and can be
    tightened later.

  - Cargo.toml: pin rust-version = '1.70' to match README's claim.
    Without this, cargo builds silently on older toolchains and surfaces
    errors as cryptic parse failures instead of a clean version-mismatch
    message. Oracle flagged this as a missing field during P2 review.

License field intentionally omitted — repo has no LICENSE file and no
publication plan yet. Add when publication becomes a goal.

Verified locally: YAML parses, cargo build --release Finished in 9.82s,
cargo test --release 79 passed, cargo clippy 0 errors.
2026-06-28 14:31:42 +08:00
dailz a6560cff6c feat(stats): wire real scale/transfer/encode timing from EncState
Oracle step 4 (option A) — give the scale_*, transfer_*, encode_* stats
fields real producers instead of misleading zeros. The fields existed in
FrameTimings and PipelineStats already; producers just weren't passing
non-zero values.

  - avhw.rs: new EncodeStages { scale_us, transfer_us, encode_us } struct.
    EncState::encode_frame (HW VAAPI path) now times the filter graph
    separately from avcodec_send_frame, returning EncodeStages. transfer_us
    is honestly 0 because the HW path never reads back to CPU.
    SwEncState::encode_frame (SW fallback path) returns EncodeStages too;
    there import_and_scale bundles GPU scale + GPU→CPU readback into one
    call, so scale_us includes transfer for SW. Documented inline.

  - state.rs: StreamingEncoder::encode_frame return type bumps from
    Result<()> to Result<EncodeStages>; wlr-screencopy path now feeds
    real per-stage timings into FrameTimings instead of just total_us.

  - state_portal.rs: HW portal path (enc.encode_frame) now extracts
    stages.scale_us / stages.transfer_us / stages.encode_us into
    FrameTimings. Removed the now-unused t_encode_start binding.

Deferred (documented):
  - state_portal.rs SW portal path (line 525) calls import_and_scale +
    enc_thread separately and bypasses SwEncState::encode_frame. To wire
    scale/transfer timing there too, either route through SwEncState or
    thread timing out of import_and_scale. Out of scope for this commit.
  - SW path lumps transfer into scale_us. Splitting requires extending
    import_and_scale's return type — left as a follow-up if operational
    need arises (current default is HW VAAPI).

Oracle audit 2026-06-28 step 4 (option A: integrate, not delete).

All 79 unit tests + 3 integration tests pass. clippy: 0 errors.
2026-06-28 14:22:45 +08:00
dailz 2ac37a1dd1 fix(stats): wire PipeWire drops, expand Display, purge dead residue
Oracle-driven P1 fix plan. Resolves the StatsSnapshot 'computed but never
consumed' debt that was silently zeroing two real diagnostic fields and
leaving a dozen more unreported.

Bug fix (Oracle step 2):
  - state_portal.rs: set_pipewire_dropped(0, 0) and set_queue_depths(0, 0)
    were hardcoded, silently discarding real PipeWire diagnostics. Now wires
    to self.cap.dropped_count() (with pw_dropped_prev delta tracking) and
    self.cap.capture_queue_depth(). The encoded side stays 0 because the
    encoder thread exposes no queue-depth API.

Display expansion (Oracle step 1):
  - stats.rs: StatsSnapshot::Display now reports 12 previously-silent fields
    paired with their existing p95/max counterparts — capture/encoded/sent
    frame counts, elapsed_secs, *_avg_ms gap timing, frame_age_avg_ms,
    per-stage import/sws/encode/total avg_ms, output_frame_bytes_p95. Each
    line of the format string maps to one operational question (cadence,
    drops, queue pressure, latency, bandwidth); layout note added.

Dead residue purge (Oracle steps 5 + 6):
  - stats.rs: removed record_over_budget method + over_budget_count field
    (no caller; total_p95_ms answers the useful question without an
    arbitrary budget threshold).
  - state.rs: removed InFlightSurface::Allocd variant (never constructed)
    and CaptureSource::alloc_frame trait method (prototype leftover; the
    sole impl in cap_wlr_screencopy.rs returned None unconditionally).
  - cap_wlr_screencopy.rs: removed the alloc_frame stub; updated the
    unit-type Frame doc to reference the asynchronicity rationale without
    the deleted method.
  - cap_portal.rs: removed redundant 'let dropped = dropped;' shadowing
    flagged by clippy::redundant_locals (line 849).

Deferred (Oracle step 4 — needs product decision):
  - scale_avg/scale_p95/transfer_avg/transfer_p95/send_wait_p95 fields
    still appear in Display but producers in the live encode path don't
    record them, so they often show misleading zeros. Either add real
    EncState timing for scale/transfer stages, or remove the fields from
    Display until then.

All 79 unit tests + 3 integration tests still pass. clippy: 0 errors.
Warning count: multiple_fields_never_read on StatsSnapshot,
method_never_used on record_over_budget/dropped_count/capture_queue_depth/
alloc_frame, variant_never_constructed on Allocd, redundant_locals on
dropped — all gone.
2026-06-28 14:15:55 +08:00
dailz 145b5d3e7e chore: design cleanup, dead-code purge, README/doc refresh
Audit-driven follow-up after the SAFETY-debt commit (Oracle steps 6-7).
End state: cargo clippy --release --all-targets still 0 errors; private_interfaces
and type_complexity warnings cleared.

Design cleanups (Oracle step 6):
  - cap_portal.rs: introduce PortalFormatInfo struct to replace the
    Rc<Cell<Option<(u32,u32,u32,u64)>>> cross-callback hand-off. Self-
    documenting struct fields replace positional tuple access at the
    format-change and process callbacks.
  - avhw.rs: import_dma_buf_to_vaapi signature collapses from 8 args
    (fd/width/height/drm_format/modifier/stride/offset) to
    (*mut AVBufferRef, &PwDmaBufFrame). Callers in avhw.rs,
    state_portal.rs, and vaapi_import_bench.rs now pass the frame by
    reference instead of unpacking 7 fields just to repack them. Drops
    the unused width parameter and the too_many_arguments(8/7) warning.
  - state.rs: visibility hygiene. EncConstructionStage and WlrHeadInfo
    downgrade pub -> pub(crate); State.stage field downgrades to
    pub(crate). These are internal state-machine types not exposed
    across the crate boundary; making them pub(crate) clears all
    private_interfaces warnings without leaking more types.

Dead-code purge (Oracle step 7):
  - transform.rs: remove unused Rect struct, transform_basis,
    screen_to_frame, fit_inside_bounds helpers and their 18 dedicated
    tests. Transform enum and transpose_if_transform_transposed remain
    (both are actively used by state.rs and avhw.rs). File shrinks
    from 409 -> 109 lines.

Repository housekeeping (Oracle step 7):
  - .gitignore: add review.json (stray review-tool output that
    regenerates per run).
  - README.md: refresh CLI table to match src/args.rs (now lists
    --backend, --no-persist, --port-as-WebRTC-signaling, --max-bitrate,
    --stats). Add capture-backend explainer + 4 new usage examples.
    Note in README points readers at src/args.rs as the authoritative
    source. Remove stale 'WebTransport, unused in MVP' description.

avhw.rs: AsRawFd import annotated with a rustc-quirk explanation — the
import triggers a false 'unused_imports' warning but E0599 if removed.
Left as-is with explanatory comment rather than chasing the lint.

All 79 remaining unit tests + 3 integration tests still pass. Cargo
build --release clean.
2026-06-28 13:58:52 +08:00
dailz 30f8fe51f2 chore: clear clippy errors, document all unsafe blocks, deny new SAFETY debt
Audit-driven cleanup pass. End state:
  - cargo clippy --release --all-targets: 0 errors (was 4)
  - undocumented_unsafe_blocks warnings: 0 (was 67)
  - Cargo.toml: undocumented_unsafe_blocks escalated warn -> deny

Clippy correctness errors fixed:
  - src/bin/{sw_encode_bench,vaapi_import_bench}.rs: receive_first_frame
    rewritten per Oracle plan with total 10s deadline + 200ms wait slice +
    while-let drain of all control events. The previous loop body always
    exited on first iteration (never_loop); the new version actually retries
    and matches production's repeated-poll semantics in state_portal.rs.
  - src/avhw.rs: hash_sampled_y_plane tests now use a row_range(row, stride,
    width) helper instead of inline stride * N. Preserves the row-index
    intent across all sibling tests without tripping erasing_op (row==0) or
    identity_op (row==1).

Machine-applicable clippy autofixes applied via 'cargo clippy --fix':
  - unnecessary_cast, manual_is_multiple_of, needless_borrows_for_generic_args
  - manual_abs_diff, derivable_impls, new_without_default
  - unnecessary_map_or, unneeded_struct_pattern, redundant_locals

webrtc_gop_formula test rewritten to wrap the (fps * 2).max(20) formula in
a runtime lambda. The previous clippy --fix pass had constant-folded the
5fps case into assert_eq!(20, 20), silently stripping the floor-case
coverage. The lambda blocks the fold while keeping the formula exercisable.

67 SAFETY comments added across 7 files (cap_portal.rs 26, sw_encode_bench
21, state_portal.rs 7, vaapi_import_bench.rs 6, avhw.rs 5, state.rs 1,
main.rs 1). Two sites carry load-bearing invariant documentation:
  - cap_portal.rs:806 process callback documents the PipeWire raw_buf
    ownership contract across all 10 exit paths (audited: every path
    correctly requeues; fd ownership via dup() is independent and also
    exactly-once closed).
  - avhw.rs:341 unsafe impl Send for EncState documents the single-thread
    exclusivity assumption referenced by AGENTS.md.

All 97 unit tests + 3 integration tests still pass; cargo build --release
finishes clean. Lint escalation to deny freezes the SAFETY baseline: any
future patch adding an unsafe block without a // SAFETY: comment will fail
clippy at compile time.
2026-06-28 13:44:27 +08:00
59 changed files with 6463 additions and 6084 deletions
+14
View File
@@ -0,0 +1,14 @@
# quick-xml is pulled in only through wayland-scanner's build-time Wayland XML
# code generation path:
#
# wayland-scanner v0.31.10 -> quick-xml v0.39.x
#
# The current wayland-scanner release requires quick-xml ^0.39, so it cannot
# accept the fixed quick-xml >=0.41.0 line yet. This project does not parse
# attacker-controlled XML at runtime through quick-xml. Remove these ignores as
# soon as wayland-scanner or the wayland-* crates release a compatible fix.
[advisories]
ignore = [
"RUSTSEC-2026-0194",
"RUSTSEC-2026-0195",
]
+118
View File
@@ -0,0 +1,118 @@
# Continuous integration for wl-webrtc.
#
# Triggered on push/PR to master. Runs the full quality gate that the recent
# audit baselined:
# - clippy: 0 errors (undocumented_unsafe_blocks is deny in Cargo.toml; other
# warnings are advisory for now).
# - build --release: integration tests in tests/integration_test.rs shell out
# to target/release/wl-webrtc, so the release binary must exist before tests
# run.
# - test --release: 79 unit + 3 integration; the 1 hardware-ignored test
# stays ignored in CI (needs Wayland session + VAAPI GPU).
# - cargo audit: separate job so a RUSTSEC advisory fails the build without
# conflating with compile errors.
#
# The job pins Linux only — the project is Wayland/VAAPI-specific and has no
# macOS/Windows story. Oracle audit 2026-06-28 P2 plan.
name: CI
on:
push:
branches: [master]
pull_request:
branches: [master]
env:
CARGO_TERM_COLOR: always
# The self-hosted act_runner network can terminate index.crates.io with a
# certificate that does not match the sparse-index hostname. Use Cargo's git
# index path in CI; GitHub access is already required for the actions above.
CARGO_REGISTRIES_CRATES_IO_PROTOCOL: git
jobs:
build-test:
name: Build + Clippy + Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Rust toolchain (stable)
uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- name: Install system dependencies
run: |
APT_OPTS=(-o Acquire::Retries=5 -o Acquire::http::Timeout=30 -o Acquire::https::Timeout=30)
sudo apt-get "${APT_OPTS[@]}" update --error-on=any
# NOTE: do NOT use --no-install-recommends for libclang-dev — on
# Debian Bookworm (the node:20-bookworm image used by act_runner
# under ubuntu-latest) the recommended toolchain bits are needed
# by bindgen. The pkg-config based deps (pipewire/wayland/etc)
# are also more reliable without the flag.
sudo apt-get "${APT_OPTS[@]}" install -y \
ffmpeg \
libavcodec-dev libavfilter-dev libavformat-dev libavutil-dev libswscale-dev libva-dev \
libwayland-dev wayland-protocols \
libdrm-dev \
libpipewire-0.3-dev \
libclang-dev clang
- name: Resolve LIBCLANG_PATH
# bindgen needs libclang on the LD path. The actual install location
# varies by Debian/Ubuntu version (llvm-14, llvm-15, ...), so we
# discover it dynamically instead of hardcoding /usr/lib/llvm-14/lib.
# Writing to $GITHUB_ENV propagates the value to subsequent steps
# inside the act_runner Docker container; the workflow-level `env:`
# block is not reliably visible there.
#
# Find pattern note: Debian Bookworm installs the runtime library as
# `libclang-14.so.1` (versioned, no plain libclang.so.* symlink),
# so a strict 'libclang.so.*' pattern returns nothing. The broader
# 'libclang*.so*' matches every bindgen-compatible filename variant:
# libclang.so, libclang-14.so, libclang.so.1, libclang-14.so.1.
run: |
set -e
LIBCL=$(find /usr -name 'libclang*.so*' 2>/dev/null | head -1)
test -n "$LIBCL" || { echo "ERROR: no libclang shared lib found under /usr"; exit 1; }
LIBDIR=$(dirname "$LIBCL")
echo "Resolved LIBCLANG_PATH=$LIBDIR (found $LIBCL)"
echo "LIBCLANG_PATH=$LIBDIR" >> "$GITHUB_ENV"
- name: Cache cargo registry + build artifacts
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-${{ hashFiles('Cargo.lock', 'Cargo.toml') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Clippy (release, all targets)
run: cargo clippy --release --all-targets
- name: Build release (required before tests)
run: cargo build --release --all-targets
- name: Test (release)
run: cargo test --release
audit:
name: Security audit (RUSTSEC)
runs-on: ubuntu-latest
# Keep separate from build-test so a vulnerability advisory fails the
# check independently of compile state.
steps:
- uses: actions/checkout@v4
- name: Install Rust toolchain (stable)
uses: dtolnay/rust-toolchain@stable
- name: Install cargo-audit
run: cargo install cargo-audit --locked
- name: Audit dependencies
run: cargo audit --deny warnings
+3
View File
@@ -21,3 +21,6 @@ Thumbs.db
.playwright-mcp/
wl-webrtc.log
webrtc-p0-success.png
# Stray review-tool output (regenerated per review run)
review.json
+4 -4
View File
@@ -9,7 +9,7 @@
- Native prerequisites are FFmpeg 6+ dev libs with VAAPI, Wayland protocols/libs, libdrm, PipeWire, and libclang. `shell.nix` provides FFmpeg/Wayland/libdrm/Mesa/libva/clang and `LIBCLANG_PATH`, but does not currently list PipeWire.
- Normal build: `cargo build`. Release binary required by README and integration tests: `cargo build --release`.
- `Cargo.toml` only warns on `clippy::undocumented_unsafe_blocks`; do not assume a broader clippy policy exists unless you add one.
- `Cargo.toml` sets `clippy::undocumented_unsafe_blocks = "deny"`; every `unsafe` block and `unsafe impl` must carry a `// SAFETY:` comment or the build fails. For `unsafe impl Send` on FFmpeg wrappers, see the convention in `src/avhw/mod.rs` — justification must be at the C-API level (atomic refcounts, libva `VADisplay` thread safety), not Rust borrow level.
## Testing and verification
@@ -24,13 +24,13 @@
- `src/main.rs` is the real entrypoint: parse `Args`, initialize tracing from `RUST_LOG` or `-v`, reject non-H.264, require `--output` or `--port`, detect backend, then run one of two loops.
- Backend detection in `src/backend_detect.rs`: explicit `--backend portal|screencopy` wins; otherwise wlr-screencopy is preferred when the Wayland global `zwlr_screencopy_manager_v1` exists, else Portal/PipeWire is used if D-Bus ScreenCast is available.
- Do not use `ashpd` for backend availability checks; `backend_detect.rs` intentionally uses raw `zbus` because `ashpd` caches a `zbus::Connection` in a global and can hang after its owning Tokio runtime is dropped.
- `src/state.rs` drives the wlroots path using a mio Wayland fd loop and `State<CapWlrScreencopy>`; `src/state_portal.rs` drives the Portal/PipeWire path through `CapPortal` frame channels.
- `src/webrtc.rs` is a small embedded HTTP/WebRTC signaling server using `str0m`; `--port 0` means file-output mode, `--port > 0` enables WebRTC mode.
- `src/state/mod.rs` drives the wlroots path using a mio Wayland fd loop and `State<CapWlrScreencopy>` (Wayland `Dispatch` impls live under `src/state/dispatch/`); `src/state_portal.rs` drives the Portal/PipeWire path through `CapPortal` frame channels (bitrate helpers and encode/webrtc thread loops are split into `src/state_portal/{bitrate,threads}.rs`). `src/cap_portal.rs` holds the `CapPortal` struct itself; its setup logic, token filesystem helpers, and PipeWire capture thread live under `src/cap_portal/`.
- `src/webrtc.rs` is a small embedded HTTP/WebRTC signaling server using `str0m`; the embedded HTML test page is in `src/webrtc/html_page.rs`. `--port 0` means file-output mode, `--port > 0` enables WebRTC mode.
## Unsafe and FFI work
- FFmpeg/VAAPI/PipeWire code relies on raw FFI and many `unsafe` blocks. Preserve nearby `// SAFETY:` explanations and add one for any new unsafe block.
- `src/avhw.rs` owns FFmpeg `AVBufferRef`/frame contexts and has explicit `unsafe impl Send`; avoid moving those wrappers across threads without rechecking the documented exclusivity assumptions.
- `src/avhw/` (split from the former `src/avhw.rs` in commit d53e881) owns FFmpeg `AVBufferRef` / frame / codec contexts. Five types (`AvHwDevCtx`, `AvHwFrameCtx`, `EncState`, `SwEncState`, `SwEncEncode`) carry `unsafe impl Send`; soundness was Oracle-audited on 2026-07-09 against FFmpeg/libva threading semantics. Moving them across threads is sound *because the C APIs use atomic refcounts*, not because of any Rust-side exclusivity — see `src/avhw/mod.rs` for the full convention.
- `CapPortal` stores the portal restore token under the user cache directory (`wl-webrtc/portal-restore-token`); use `--no-persist` when manually testing fresh authorization behavior.
## Useful manual commands
Generated
+2 -2
View File
@@ -94,9 +94,9 @@ dependencies = [
[[package]]
name = "anyhow"
version = "1.0.102"
version = "1.0.103"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
checksum = "2a4385e2e34eb35d6b3efe798b9eb88096925d87726c0798709bf56d9ed84af3"
[[package]]
name = "arrayvec"
+7 -1
View File
@@ -2,6 +2,12 @@
name = "wl-webrtc"
version = "0.1.0"
edition = "2021"
# MSRV pinned to 1.87 to match the actual API floor — the codebase uses
# u32::is_multiple_of (1.87) and Option::is_none_or (1.82) introduced by
# clippy autofixes. README's Prerequisites section mirrors this. Bumping
# this floor requires checking clippy::incompatible_msrv against the new
# value.
rust-version = "1.87"
description = "Wayland screen capture and encoding tool"
[dependencies]
@@ -33,4 +39,4 @@ dirs = "6"
tempfile = "3.27.0"
[lints.clippy]
undocumented_unsafe_blocks = "warn"
undocumented_unsafe_blocks = "deny"
+30 -3
View File
@@ -4,7 +4,7 @@ Wayland screen capture and encoding tool.
## Prerequisites
- **Rust toolchain** (1.70+): `rustup default stable`
- **Rust toolchain** (1.87+; MSRV pinned to match `u32::is_multiple_of` / `Option::is_none_or` usage): `rustup default stable`
- **FFmpeg 6.0+** dev libraries with VAAPI support:
- Arch: `pacman -S ffmpeg`
- Ubuntu/Debian: `apt install libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libva-dev`
@@ -38,19 +38,46 @@ wl-webrtc --output output.mp4 --drm-device /dev/dri/renderD128
# Verbose mode
wl-webrtc --output output.mp4 -v
# WebRTC streaming mode (HTTP signaling server)
wl-webrtc --port 8080 -v
# Force a fresh portal authorization dialog (ignore saved restore token)
wl-webrtc --output output.mp4 --no-persist
# Pin the capture backend instead of auto-detecting
wl-webrtc --output output.mp4 --backend portal # or: --backend screencopy
```
## CLI Arguments
> `src/args.rs` is the authoritative source. Run `wl-webrtc --help` for the live list.
| Argument | Default | Description |
|---|---|---|
| `-o`, `--output` | (required) | Output file path (e.g., output.mp4) |
| `-o`, `--output` | (optional) | Output file path (e.g. output.mp4). Optional when using `--port` for WebRTC mode. |
| `--output-name` | auto | Wayland output name to capture |
| `--fps` | 30 | Target frames per second |
| `--codec` | h264 | Video codec (h264 only for MVP) |
| `--hw-accel` | vaapi | Hardware acceleration method |
| `--drm-device` | auto | DRM render device path |
| `--bitrate` | auto | Target bitrate in bps |
| `--max-bitrate` | 8000000 | Max bitrate cap for WebRTC mode (caps BWE escalation; no effect in MP4 mode) |
| `--gop-size` | auto | Group of Pictures size |
| `-v`, `--verbose` | false | Enable verbose logging |
| `--port` | 0 | WebTransport server port (unused in MVP) |
| `--backend` | auto | Capture backend: `screencopy` (wlroots) or `portal` (KWin/KDE). Auto-detected if omitted. |
| `--port` | 0 | WebRTC HTTP signaling server port. `0` keeps MP4 file output mode. |
| `--no-persist` | false | Force re-authorization (ignore saved portal restore token) |
| `--stats` | false | Print per-second pipeline statistics for stutter diagnosis |
## Capture backends
The tool supports two Wayland capture backends, auto-detected by default:
- **wlr-screencopy** (preferred when `zwlr_screencopy_manager_v1` is advertised):
works on wlroots-based compositors (Sway, Hyprland, etc.).
- **XDG Portal / PipeWire** (fallback when D-Bus ScreenCast is available):
works on KWin/KDE and any compositor that implements the XDG Desktop Portal
screen-cast protocol. The first run shows an authorization dialog; a restore
token is cached under `wl-webrtc/portal-restore-token` so subsequent runs
don't re-prompt (use `--no-persist` to force a fresh authorization).
-2169
View File
File diff suppressed because it is too large Load Diff
+138
View File
@@ -0,0 +1,138 @@
use std::ffi::CString;
use std::path::Path;
use std::ptr;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::util::ff_err;
pub struct AvHwDevCtx {
ptr: *mut ffi::AVBufferRef,
}
// SAFETY: AVBufferRef's refcount is atomic (atomic_uint in libavutil/buffer.c);
// av_buffer_ref / av_buffer_unref are safe to call concurrently from different
// threads on the same buffer. The underlying AVHWDeviceContext (VAAPI VADisplay)
// is designed by FFmpeg/libva to be shared across codec and filter contexts,
// including across FFmpeg-internal codec threads. Raw refs returned by
// ref_clone() may outlive this wrapper and be consumed by other threads; this
// is the intended usage pattern and is sound because refcount management is
// atomic. The &mut self on Rust methods is an API convenience, not the basis
// for soundness.
unsafe impl Send for AvHwDevCtx {}
impl AvHwDevCtx {
pub fn new_vaapi(drm_device: &Path) -> Result<Self> {
let device_cstr = CString::new(drm_device.to_str().unwrap())?;
let mut p: *mut ffi::AVBufferRef = ptr::null_mut();
// SAFETY: device_cstr is a valid C string for the duration of the call;
// p is a valid out-pointer that FFmpeg initializes on success.
let ret = unsafe {
ffi::av_hwdevice_ctx_create(
&mut p,
ffi::AVHWDeviceType::AV_HWDEVICE_TYPE_VAAPI,
device_cstr.as_ptr(),
ptr::null_mut(),
0,
)
};
if ret < 0 {
bail!(
"Failed to create VAAPI device context from {}: {}",
drm_device.display(),
ff_err(ret)
);
}
Ok(Self { ptr: p })
}
pub fn as_ptr(&self) -> *mut ffi::AVBufferRef {
self.ptr
}
pub fn ref_clone(&self) -> *mut ffi::AVBufferRef {
// SAFETY: av_buffer_ref atomically increments refcount and returns a new ref.
unsafe { ffi::av_buffer_ref(self.ptr) }
}
}
impl Drop for AvHwDevCtx {
fn drop(&mut self) {
if !self.ptr.is_null() {
// SAFETY: av_buffer_unref decrements refcount; frees the buffer when it hits zero.
unsafe { ffi::av_buffer_unref(&mut self.ptr) };
}
}
}
pub struct AvHwFrameCtx {
ptr: *mut ffi::AVBufferRef,
}
// SAFETY: AVBufferRef's refcount is atomic (see AvHwDevCtx). The underlying
// AVHWFramesContext allocates from an AVBufferPool, whose get/put operations
// are atomic and thread-safe. av_hwframe_get_buffer and av_hwframe_transfer_data
// are safe to call concurrently on distinct AVFrames. Cloned refs are typically
// attached to AVCodecContext.hw_frames_ctx and accessed by FFmpeg-internal codec
// threads; this is the designed usage. The &mut self on Rust methods is not the
// basis for soundness.
unsafe impl Send for AvHwFrameCtx {}
impl AvHwFrameCtx {
fn new_inner(hw_dev: &AvHwDevCtx, w: u32, h: u32, sw_fmt: ff::format::Pixel) -> Result<Self> {
// SAFETY: hw_dev is a live AVHWDeviceContext; FFmpeg returns either a valid
// frames context ref or null (checked below).
let mut p = unsafe { ffi::av_hwframe_ctx_alloc(hw_dev.as_ptr()) };
if p.is_null() {
bail!("av_hwframe_ctx_alloc returned null");
}
// SAFETY: p is a valid AVBufferRef from av_hwframe_ctx_alloc.
// Its .data field points to an AVHWFramesContext that we must configure.
unsafe {
let fc = (*p).data as *mut ffi::AVHWFramesContext;
(*fc).format = ff::format::Pixel::VAAPI.into();
(*fc).sw_format = sw_fmt.into();
(*fc).width = w as i32;
(*fc).height = h as i32;
(*fc).initial_pool_size = 4;
}
// SAFETY: p is a valid AVHWFramesContext ref configured above and not yet
// transferred or freed.
let ret = unsafe { ffi::av_hwframe_ctx_init(p) };
if ret < 0 {
// SAFETY: p is valid but init failed; clean up.
unsafe { ffi::av_buffer_unref(&mut p) };
bail!("av_hwframe_ctx_init failed: {}", ff_err(ret));
}
Ok(Self { ptr: p })
}
pub fn for_capture(
hw_dev: &AvHwDevCtx,
w: u32,
h: u32,
sw_fmt: ff::format::Pixel,
) -> Result<Self> {
Self::new_inner(hw_dev, w, h, sw_fmt)
}
pub fn as_ptr(&self) -> *mut ffi::AVBufferRef {
self.ptr
}
pub fn ref_clone(&self) -> *mut ffi::AVBufferRef {
// SAFETY: av_buffer_ref atomically increments refcount and returns a new ref.
unsafe { ffi::av_buffer_ref(self.ptr) }
}
}
impl Drop for AvHwFrameCtx {
fn drop(&mut self) {
if !self.ptr.is_null() {
// SAFETY: av_buffer_unref decrements refcount; frees when zero.
unsafe { ffi::av_buffer_unref(&mut self.ptr) };
}
}
}
+125
View File
@@ -0,0 +1,125 @@
use std::mem;
// AsRawFd is required by `frame.fd.as_raw_fd()` below but rustc emits a false
// "unused_imports" warning because OwnedFd also has an inherent `as_raw_fd`.
// E0599 if removed -> must stay; warning is a known rustc quirk.
use std::os::fd::AsRawFd;
use std::os::raw::c_void;
use std::path::Path;
use std::ptr;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use crate::cap_portal::PwDmaBufFrame;
use super::{ff_err, AvHwDevCtx, AvHwFrameCtx};
/// Test whether `drm_device` can import the PipeWire DMA-BUF frame via VAAPI.
pub fn test_dma_buf_import(drm_device: &Path, frame: &PwDmaBufFrame) -> Result<()> {
let hw_dev = AvHwDevCtx::new_vaapi(drm_device)?;
let frames =
AvHwFrameCtx::for_capture(&hw_dev, frame.width, frame.height, ff::format::Pixel::BGRA)?;
// SAFETY: frames is a live VAAPI frames context; frame carries valid DMA-BUF metadata.
unsafe { import_dma_buf_to_vaapi(frames.as_ptr(), frame) }?;
Ok(())
}
/// Import a DMA-BUF into a VAAPI hardware frame via zero-copy `av_hwframe_map`.
///
/// # Safety
/// Imports a DMA-BUF frame into a VAAPI hardware frame pool for GPU-side processing.
///
/// Takes the negotiated format/geometry from `frame` (a `PwDmaBufFrame` from
/// PipeWire capture) plus the target `frames_ctx` (VAAPI frame pool from
/// `AvHwFrameCtx`) and returns an `ff::frame::Video` whose data[3] points to
/// the hardware frame.
///
/// # Safety
///
/// - `frames_ctx` must point to an initialized AVHWCramesContext for VAAPI
/// - `frame.fd` must be a valid DMA-BUF file descriptor
pub unsafe fn import_dma_buf_to_vaapi(
frames_ctx: *mut ffi::AVBufferRef,
frame: &PwDmaBufFrame,
) -> Result<ff::frame::Video> {
let duped_fd = libc::dup(frame.fd.as_raw_fd());
if duped_fd < 0 {
bail!("dup(fd) failed: {}", std::io::Error::last_os_error());
}
let mut desc: ffi::AVDRMFrameDescriptor = mem::zeroed();
desc.nb_objects = 1;
desc.objects[0].fd = duped_fd;
desc.objects[0].size = (frame.height as usize) * (frame.stride as usize);
desc.objects[0].format_modifier = frame.modifier;
desc.nb_layers = 1;
desc.layers[0].format = frame.format;
desc.layers[0].nb_planes = 1;
desc.layers[0].planes[0].object_index = 0;
desc.layers[0].planes[0].offset = frame.offset as isize;
desc.layers[0].planes[0].pitch = frame.stride as isize;
let desc_box = Box::new(desc);
let desc_ptr = Box::into_raw(desc_box);
let buf_ref = ffi::av_buffer_create(
desc_ptr as *mut u8,
std::mem::size_of::<ffi::AVDRMFrameDescriptor>(),
Some(cleanup_drm_descriptor),
ptr::null_mut(),
0,
);
if buf_ref.is_null() {
let desc_box = Box::from_raw(desc_ptr);
libc::close(desc_box.objects[0].fd);
bail!("av_buffer_create returned null for DRM descriptor");
}
let mut src = ff::frame::Video::empty();
{
let sp = src.as_mut_ptr();
(*sp).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME as i32;
(*sp).width = frame.width as i32;
(*sp).height = frame.height as i32;
(*sp).data[0] = (*buf_ref).data;
(*sp).buf[0] = buf_ref;
}
let mut dst = ff::frame::Video::empty();
// SAFETY: frames_ctx is guaranteed by this unsafe function's contract to be a
// valid initialized VAAPI frames context; we set format/hw_frames_ctx on a
// freshly allocated dst frame.
unsafe {
let dp = dst.as_mut_ptr();
(*dp).format = ffi::AVPixelFormat::AV_PIX_FMT_VAAPI as i32;
(*dp).hw_frames_ctx = ffi::av_buffer_ref(frames_ctx);
if (*dp).hw_frames_ctx.is_null() {
bail!("av_buffer_ref(frames_ctx) returned null");
}
}
// SAFETY: src and dst are initialized AVFrames; dst has a valid hw_frames_ctx
// ref and av_hwframe_map fills dst from src.
let ret = unsafe {
ffi::av_hwframe_map(
dst.as_mut_ptr(),
src.as_ptr(),
ffi::AV_HWFRAME_MAP_READ as i32,
)
};
if ret < 0 {
bail!("av_hwframe_map failed: {}", ff_err(ret));
}
Ok(dst)
}
unsafe extern "C" fn cleanup_drm_descriptor(_opaque: *mut c_void, data: *mut u8) {
let desc = data as *mut ffi::AVDRMFrameDescriptor;
if !desc.is_null() && (*desc).nb_objects > 0 && (*desc).objects[0].fd >= 0 {
libc::close((*desc).objects[0].fd);
}
let _ = Box::from_raw(data as *mut ffi::AVDRMFrameDescriptor);
}
+305
View File
@@ -0,0 +1,305 @@
use std::mem;
use std::ptr;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use ffmpeg_next::packet::Mut as _;
use super::encode_output::{self, FrameOutput, PacketOutput};
use super::hash::hash_sampled_y_plane;
use super::{
ff_err, BitrateCommand, CpuNv12Frame, EncodeOutcome, ResolutionChange, SwEncodeTiming,
};
pub struct SwEncEncode {
pub(super) sws_ctx: *mut ffi::SwsContext,
pub(super) enc_video: ff::codec::encoder::video::Video,
pub(super) output: Option<FrameOutput>,
pub(super) yuv_frame: *mut ffi::AVFrame,
pub(super) last_frame_hash: u64,
pub(super) frame_count: u64,
pub(super) starting_timestamp: Option<i64>,
pub(super) frames_written: bool,
pub(super) webrtc_disconnected: bool,
pub(super) webrtc_paused: Option<Arc<AtomicBool>>,
pub(super) bitrate_rx: crossbeam_channel::Receiver<BitrateCommand>,
pub(super) resolution_rx: crossbeam_channel::Receiver<ResolutionChange>,
pub(super) enc_width: u32,
pub(super) enc_height: u32,
pub(super) fps: u32,
pub(super) bitrate: u64,
pub(super) gop_size: u32,
/// Set true when WebRTC requests a keyframe. Forces the next frame to
/// `AV_PICTURE_TYPE_I` and bypasses the dedup hash check. Cleared only
/// after `avcodec_send_frame` accepts the forced frame.
pub(super) force_keyframe_pending: bool,
/// Last per-frame timing snapshot. Reset to `Default` at the start of
/// every `encode_cpu_frame` call (even on early returns) so stale values
/// from a previous frame can never leak out.
pub(super) last_timing: SwEncodeTiming,
/// Capture time of the frame currently being encoded. Saved from the
/// input `CpuNv12Frame` so `drain_encoder` can propagate it into the
/// emitted `EncodedH264Frame` for the frame_age stat (issue #20).
pub(super) last_capture_time: Option<Instant>,
}
/// WebRTC media clock frequency in Hz. Matches RTP clock for video (RFC 3551).
/// Used as encoder time_base denominator for WebRTC mode (1/90000) so that
/// PTS values directly become RTP timestamps with microsecond precision.
/// MP4 mode keeps 1/fps time_base for file output simplicity.
pub const WEBRTC_RTP_CLOCK_HZ: i128 = 90_000;
// SAFETY: SwEncEncode is moved to a single encode thread and accessed only there
// via &mut self. SwsContext, AVFrame, and AVCodecContext are NOT thread-safe for
// concurrent access but are Send-sound under single-thread exclusive use, which
// the encode worker invariant provides. crossbeam Receiver and Arc<AtomicBool>
// are Send by design.
unsafe impl Send for SwEncEncode {}
impl SwEncEncode {
pub fn flush(&mut self) -> Result<()> {
// SAFETY: Sending a null frame flushes the opened software encoder;
// no frame data is dereferenced. enc_video is exclusively borrowed via &mut self.
unsafe {
let ret = ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), ptr::null());
if ret < 0 && ret != ffi::AVERROR_EOF {
bail!("software encoder flush send failed: {}", ff_err(ret));
}
}
let start_ts = self.starting_timestamp.unwrap_or(0);
let _ = self.drain_encoder(start_ts)?;
Ok(())
}
pub fn take_timing(&mut self) -> SwEncodeTiming {
mem::take(&mut self.last_timing)
}
pub fn encode_cpu_frame(&mut self, frame: &CpuNv12Frame) -> Result<EncodeOutcome> {
self.last_timing = SwEncodeTiming::default();
// Save capture_time so drain_encoder can propagate it into the
// EncodedH264Frame emitted via the WebRTC channel (issue #20).
self.last_capture_time = Some(frame.capture_time);
if self.webrtc_disconnected {
return Ok(EncodeOutcome::SkippedDisconnected);
}
// Must drain before the stride check: the import thread emits
// ResolutionChange before the new (smaller-stride) frame arrives.
while let Ok(cmd) = self.bitrate_rx.try_recv() {
match cmd {
BitrateCommand::UpdateBitrate { target_bps } => {
// #23 defensive guardrail: clamp to reasonable max even if policy layer
// is bypassed. 50 Mbps is a hard ceiling; primary cap is enforced in
// state_portal.rs webrtc_thread_loop via --max-bitrate flag.
const ENCODER_BITRATE_HARD_CAP: u64 = 50_000_000;
let target_bps = target_bps.min(ENCODER_BITRATE_HARD_CAP);
tracing::info!(target_bps, "updating encoder bitrate from BWE feedback");
self.bitrate = target_bps;
// SAFETY: enc_video is an opened AVCodecContext exclusively owned by &mut self.
unsafe {
let ctx = self.enc_video.as_mut_ptr();
(*ctx).bit_rate = target_bps as i64;
}
}
BitrateCommand::UpdateResolution { .. } => {}
BitrateCommand::ForceKeyframe => {
self.force_keyframe_pending = true;
tracing::debug!("encode thread: ForceKeyframe requested");
}
}
}
let force_this_frame = self.force_keyframe_pending;
while let Ok(change) = self.resolution_rx.try_recv() {
self.recreate_encoder(change.width, change.height)?;
}
if frame.y_stride < self.enc_width as usize || frame.uv_stride < self.enc_width as usize {
bail!("CPU NV12 frame stride is smaller than encoder width");
}
if let Some(ref paused) = self.webrtc_paused {
if paused.load(Ordering::Relaxed) {
return Ok(EncodeOutcome::SkippedPaused);
}
}
let width = self.enc_width as usize;
let height = self.enc_height as usize;
let required_y_len = frame.y_stride * height.saturating_sub(1) + width;
if frame.y_data.len() < required_y_len {
bail!("CPU NV12 frame Y plane is smaller than encoder dimensions");
}
let frame_index = self.frame_count;
self.frame_count = self.frame_count.saturating_add(1);
let current_hash = hash_sampled_y_plane(&frame.y_data, width, height, frame.y_stride);
let force_gop_frame =
self.gop_size > 0 && frame_index.is_multiple_of(u64::from(self.gop_size));
if frame_index > 0
&& !force_gop_frame
&& !force_this_frame
&& current_hash == self.last_frame_hash
{
tracing::debug!(frame_index, "skipping duplicate frame");
self.last_frame_hash = current_hash;
return Ok(EncodeOutcome::SkippedDuplicate);
}
self.last_frame_hash = current_hash;
let sws_start = Instant::now();
// SAFETY: yuv_frame is an owned reusable YUV420P frame at the same dimensions as sw_nv12;
// sws_ctx was created for NV12 -> YUV420P with no resize, so sws_scale only converts format.
unsafe {
let ret = ffi::av_frame_make_writable(self.yuv_frame);
if ret < 0 {
bail!("av_frame_make_writable failed: {}", ff_err(ret));
}
let src_slices = [
frame.y_data.as_ptr(),
frame.uv_data.as_ptr(),
ptr::null(),
ptr::null(),
];
let src_strides = [frame.y_stride as i32, frame.uv_stride as i32, 0, 0];
let scaled = ffi::sws_scale(
self.sws_ctx,
src_slices.as_ptr(),
src_strides.as_ptr(),
0,
self.enc_height as i32,
(*self.yuv_frame).data.as_ptr() as *mut *mut u8,
(*self.yuv_frame).linesize.as_ptr(),
);
if scaled < 0 {
bail!("sws_scale failed for software encoder: {scaled}");
}
}
let sws_us = sws_start.elapsed().as_micros() as u64;
let pts = frame.pts;
if self.starting_timestamp.is_none() {
self.starting_timestamp = Some(pts);
}
let start_ts = self.starting_timestamp.unwrap_or(0);
let enc_start = Instant::now();
// SAFETY: yuv_frame is initialized, writable, and matches the opened encoder format.
// pict_type is reset every frame: the AVFrame is reused, so without resetting to NONE
// a previously-forced I-type would leak into subsequent P-frames. With forced-idr=1
// set on the encoder, AV_PICTURE_TYPE_I produces a true IDR NALU.
unsafe {
(*self.yuv_frame).pts = pts;
(*self.yuv_frame).pict_type = if force_this_frame {
ffi::AVPictureType::AV_PICTURE_TYPE_I
} else {
ffi::AVPictureType::AV_PICTURE_TYPE_NONE
};
let ret = ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), self.yuv_frame);
if ret < 0 {
bail!(
"avcodec_send_frame failed for software encoder: {}",
ff_err(ret)
);
}
}
if force_this_frame {
self.force_keyframe_pending = false;
}
let output_bytes = self.drain_encoder(start_ts)?;
let encode_us = enc_start.elapsed().as_micros() as u64;
self.last_timing = SwEncodeTiming {
sws_us,
encode_us,
output_bytes,
};
Ok(EncodeOutcome::Encoded)
}
pub(super) fn write_trailer_if_needed(&mut self) -> Result<()> {
if self.frames_written {
if let Some(FrameOutput::Muxer(ref mut octx)) = self.output {
octx.write_trailer()
.map_err(|e| anyhow::anyhow!("Failed to write trailer: {e}"))?;
}
}
Ok(())
}
fn drain_encoder(&mut self, start_ts: i64) -> Result<usize> {
let mut total_bytes = 0usize;
loop {
let mut pkt = ff::Packet::empty();
// SAFETY: enc_video is an open encoder; pkt is writable packet storage.
let ret = unsafe {
ffi::avcodec_receive_packet(self.enc_video.as_mut_ptr(), pkt.as_mut_ptr())
};
if ret < 0 {
if ret == ffi::AVERROR(ffi::EAGAIN) || ret == ffi::AVERROR_EOF {
break;
}
bail!("avcodec_receive_packet failed: {}", ff_err(ret));
}
// Count encoded bytes produced before the Muxer/Channel match to
// avoid branch duplication and handle multi-packet drain correctly.
// SAFETY: pkt was just filled by a successful avcodec_receive_packet;
// the size field is valid and initialized.
let pkt_size = unsafe { (*pkt.as_mut_ptr()).size };
if pkt_size > 0 {
total_bytes += pkt_size as usize;
}
match self.output {
Some(FrameOutput::Muxer(ref mut octx)) => {
encode_output::write_muxer_packet(
&mut pkt,
octx,
self.enc_video.time_base(),
start_ts,
)?;
self.frames_written = true;
}
Some(FrameOutput::Channel(ref tx))
if encode_output::send_channel_packet(
&mut pkt,
tx,
start_ts,
self.last_capture_time.unwrap_or_else(Instant::now),
)? == PacketOutput::Disconnected =>
{
self.webrtc_disconnected = true;
break;
}
Some(FrameOutput::Channel(_)) => {}
None => {}
}
}
Ok(total_bytes)
}
}
impl Drop for SwEncEncode {
fn drop(&mut self) {
if !self.sws_ctx.is_null() {
// SAFETY: sws_ctx is owned by this state and was returned by sws_getContext.
unsafe { ffi::sws_freeContext(self.sws_ctx) };
self.sws_ctx = ptr::null_mut();
}
if !self.yuv_frame.is_null() {
// SAFETY: yuv_frame is owned by this state and was allocated by av_frame_alloc.
unsafe { ffi::av_frame_free(&mut self.yuv_frame) };
}
}
}
+133
View File
@@ -0,0 +1,133 @@
use std::path::Path;
use std::ptr;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use anyhow::Result;
use ffmpeg_next::ffi;
use super::encode_output::FrameOutput;
use super::software::{
alloc_yuv420p_frame, create_nv12_to_yuv420p_sws, create_software_h264_encoder,
create_software_h264_muxer,
};
use super::{BitrateCommand, EncodedH264Frame, ResolutionChange, SwEncEncode, SwEncodeTiming};
impl SwEncEncode {
#[allow(clippy::too_many_arguments)]
pub(super) fn new_muxer(
output_path: &Path,
enc_width: u32,
enc_height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
) -> Result<Self> {
let sws_ctx = create_nv12_to_yuv420p_sws(enc_width, enc_height)?;
let (enc_video, octx) =
create_software_h264_muxer(output_path, enc_width, enc_height, fps, bitrate, gop_size)?;
let yuv_frame = alloc_yuv420p_frame(enc_width, enc_height)?;
let (dummy_tx, bitrate_rx) = crossbeam_channel::bounded(1);
drop(dummy_tx);
let (dummy_resolution_tx, resolution_rx) = crossbeam_channel::bounded(1);
drop(dummy_resolution_tx);
Ok(Self {
sws_ctx,
enc_video,
output: Some(FrameOutput::Muxer(octx)),
yuv_frame,
last_frame_hash: 0,
frame_count: 0,
starting_timestamp: None,
frames_written: false,
webrtc_disconnected: false,
webrtc_paused: None,
bitrate_rx,
resolution_rx,
enc_width,
enc_height,
fps,
bitrate,
gop_size,
force_keyframe_pending: false,
last_timing: SwEncodeTiming::default(),
last_capture_time: None,
})
}
#[allow(clippy::too_many_arguments)]
pub fn new_webrtc(
enc_width: u32,
enc_height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
tx: crossbeam_channel::Sender<EncodedH264Frame>,
webrtc_paused: Arc<AtomicBool>,
bitrate_rx: crossbeam_channel::Receiver<BitrateCommand>,
resolution_rx: crossbeam_channel::Receiver<ResolutionChange>,
) -> Result<Self> {
let sws_ctx = create_nv12_to_yuv420p_sws(enc_width, enc_height)?;
let enc_video =
create_software_h264_encoder(enc_width, enc_height, fps, bitrate, gop_size)?;
let yuv_frame = alloc_yuv420p_frame(enc_width, enc_height)?;
Ok(Self {
sws_ctx,
enc_video,
output: Some(FrameOutput::Channel(tx)),
yuv_frame,
last_frame_hash: 0,
frame_count: 0,
starting_timestamp: None,
frames_written: false,
webrtc_disconnected: false,
webrtc_paused: Some(webrtc_paused),
bitrate_rx,
resolution_rx,
enc_width,
enc_height,
fps,
bitrate,
gop_size,
force_keyframe_pending: false,
last_timing: SwEncodeTiming::default(),
last_capture_time: None,
})
}
pub(super) fn recreate_encoder(&mut self, width: u32, height: u32) -> Result<()> {
if width == self.enc_width && height == self.enc_height {
return Ok(());
}
tracing::info!(
from = format_args!("{}x{}", self.enc_width, self.enc_height),
to = format_args!("{}x{}", width, height),
"recreating WebRTC software encoder for resolution change"
);
if !self.sws_ctx.is_null() {
// SAFETY: sws_ctx is owned exclusively by self and will be replaced below.
unsafe { ffi::sws_freeContext(self.sws_ctx) };
self.sws_ctx = ptr::null_mut();
}
if !self.yuv_frame.is_null() {
// SAFETY: yuv_frame is owned exclusively by self and will be replaced below.
unsafe { ffi::av_frame_free(&mut self.yuv_frame) };
}
self.sws_ctx = create_nv12_to_yuv420p_sws(width, height)?;
self.enc_video =
create_software_h264_encoder(width, height, self.fps, self.bitrate, self.gop_size)?;
self.yuv_frame = alloc_yuv420p_frame(width, height)?;
self.enc_width = width;
self.enc_height = height;
self.last_frame_hash = 0;
self.frame_count = 0;
self.force_keyframe_pending = true;
Ok(())
}
}
+99
View File
@@ -0,0 +1,99 @@
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::packet::Mut as _;
use super::EncodedH264Frame;
pub enum FrameOutput {
Muxer(ff::format::context::Output),
Channel(crossbeam_channel::Sender<EncodedH264Frame>),
}
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub(super) enum PacketOutput {
Written,
Dropped,
Disconnected,
}
pub(super) fn write_muxer_packet(
pkt: &mut ff::Packet,
octx: &mut ff::format::context::Output,
enc_tb: ff::Rational,
start_ts: i64,
) -> Result<()> {
// SAFETY: muxer output was created with stream 0 during setup; streams
// is non-null and stream 0 remains owned by the format context.
let stream_tb = unsafe {
let fmt = *octx.as_ptr();
if fmt.nb_streams == 0 || fmt.streams.is_null() {
bail!("no streams in output context");
}
let st = *fmt.streams.add(0);
ff::Rational::from((*st).time_base)
};
pkt.rescale_ts(enc_tb, stream_tb);
if let Some(pts) = pkt.pts() {
pkt.set_pts(Some(pts - start_ts));
}
if let Some(dts) = pkt.dts() {
pkt.set_dts(Some(dts - start_ts));
}
pkt.set_stream(0);
pkt.write_interleaved(octx)
.map_err(|e| anyhow::anyhow!("Failed to write packet: {e}"))
}
pub(super) fn send_channel_packet(
pkt: &mut ff::Packet,
tx: &crossbeam_channel::Sender<EncodedH264Frame>,
start_ts: i64,
capture_time: Instant,
) -> Result<PacketOutput> {
// SAFETY: pkt is a valid AVPacket just filled by avcodec_receive_packet;
// this copies fields for read-only inspection before pkt is dropped.
let raw = unsafe { *pkt.as_mut_ptr() };
if raw.size <= 0 || raw.data.is_null() {
return Ok(PacketOutput::Dropped);
}
// SAFETY: `pkt` is a valid AVPacket just filled by a successful
// `avcodec_receive_packet` call. We checked `size > 0` and `data` is
// non-null, so `data` points to `size` initialized bytes owned by the
// packet. `u8` has alignment 1, and the slice is copied into a Vec before
// the packet is unreffed.
let data = unsafe { std::slice::from_raw_parts(raw.data, raw.size as usize) };
let pts_ticks = match pkt.pts() {
Some(p) => p - start_ts,
None => {
tracing::warn!("encoder produced packet without PTS, dropping");
return Ok(PacketOutput::Dropped);
}
};
match tx.try_send(EncodedH264Frame {
data: data.to_vec(),
pts_ticks,
capture_time,
}) {
Ok(()) => Ok(PacketOutput::Written),
Err(crossbeam_channel::TrySendError::Full(frame)) => {
tracing::warn!(
"WebRTC channel full, dropping frame: {} bytes lost",
frame.data.len()
);
Ok(PacketOutput::Dropped)
}
Err(crossbeam_channel::TrySendError::Disconnected(frame)) => {
tracing::warn!(
"WebRTC channel disconnected: {} bytes lost",
frame.data.len()
);
Ok(PacketOutput::Disconnected)
}
}
}
+174
View File
@@ -0,0 +1,174 @@
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use crate::transform::Transform;
use super::{ff_err, AvHwDevCtx, AvHwFrameCtx};
#[allow(clippy::too_many_arguments)]
pub(super) fn build_swenc_filter_graph(
hw_dev: &AvHwDevCtx,
frames_rgb: &AvHwFrameCtx,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
) -> Result<ff::filter::Graph> {
let mut graph = ff::filter::Graph::new();
let buffersrc =
ff::filter::find("buffer").ok_or_else(|| anyhow::anyhow!("filter 'buffer' not found"))?;
let buffersink = ff::filter::find("buffersink")
.ok_or_else(|| anyhow::anyhow!("filter 'buffersink' not found"))?;
let scale_vaapi = ff::filter::find("scale_vaapi")
.ok_or_else(|| anyhow::anyhow!("filter 'scale_vaapi' not found"))?;
// FFmpeg 8.0+ rejects VAAPI pix_fmt in buffer args before hw_frames_ctx is attached.
// Use a SW placeholder, then override format/hw_frames_ctx with av_buffersrc_parameters_set.
let args = format!(
"video_size={}x{}:pix_fmt=bgra:time_base=1/{fps}:pixel_aspect=1/1",
width, height,
);
let mut src_ctx = graph.add(&buffersrc, "in", &args)?;
// SAFETY: av_buffersrc_parameters_alloc returns newly allocated parameters
// or null, which is checked below.
let par = unsafe { ffi::av_buffersrc_parameters_alloc() };
if par.is_null() {
bail!("av_buffersrc_parameters_alloc returned null");
}
// SAFETY: par and src_ctx are valid; frames_rgb.ref_clone returns an owned hw_frames_ctx ref
// that buffersrc consumes on successful parameter set.
unsafe {
(*par).format = Into::<ffi::AVPixelFormat>::into(ff::format::Pixel::VAAPI) as i32;
(*par).width = width as i32;
(*par).height = height as i32;
(*par).time_base = ffi::AVRational {
num: 1,
den: fps as i32,
};
(*par).hw_frames_ctx = frames_rgb.ref_clone();
let ret = ffi::av_buffersrc_parameters_set(src_ctx.as_mut_ptr(), par);
ffi::av_free(par as *mut _);
if ret < 0 {
bail!("av_buffersrc_parameters_set failed: {}", ff_err(ret));
}
}
let mut scale_ctx = graph.add(
&scale_vaapi,
"scale",
&format!("{enc_width}:{enc_height}:format=nv12"),
)?;
// SAFETY: scale_vaapi keeps a ref-counted device context while the graph is alive.
unsafe {
(*scale_ctx.as_mut_ptr()).hw_device_ctx = hw_dev.ref_clone();
}
let mut sink_ctx = graph.add(&buffersink, "out", "")?;
src_ctx.link(0, &mut scale_ctx, 0);
scale_ctx.link(0, &mut sink_ctx, 0);
graph
.validate()
.map_err(|e| anyhow::anyhow!("software GPU filter graph validation failed: {e}"))?;
Ok(graph)
}
pub(super) fn build_filter_graph(
hw_dev: &AvHwDevCtx,
frames_rgb: &AvHwFrameCtx,
width: u32,
height: u32,
fps: u32,
transform: Transform,
) -> Result<ff::filter::Graph> {
let mut graph = ff::filter::Graph::new();
let buffersrc =
ff::filter::find("buffer").ok_or_else(|| anyhow::anyhow!("filter 'buffer' not found"))?;
let buffersink = ff::filter::find("buffersink")
.ok_or_else(|| anyhow::anyhow!("filter 'buffersink' not found"))?;
let scale_vaapi = ff::filter::find("scale_vaapi")
.ok_or_else(|| anyhow::anyhow!("filter 'scale_vaapi' not found"))?;
// buffersrc - use AVBufferSrcParameters to set hw_frames_ctx properly.
let args = format!(
"video_size={}x{}:pix_fmt={}:time_base=1/{fps}:pixel_aspect=1/1",
width,
height,
Into::<ffi::AVPixelFormat>::into(ff::format::Pixel::VAAPI) as i32,
);
let mut src_ctx = graph.add(&buffersrc, "in", &args)?;
// SAFETY: av_buffersrc_parameters_alloc allocates params for the buffersrc.
let par = unsafe { ffi::av_buffersrc_parameters_alloc() };
if par.is_null() {
bail!("av_buffersrc_parameters_alloc returned null");
}
// SAFETY: Set hw_frames_ctx on the buffersrc parameters, then apply.
unsafe {
(*par).format = Into::<ffi::AVPixelFormat>::into(ff::format::Pixel::VAAPI) as i32;
(*par).width = width as i32;
(*par).height = height as i32;
(*par).time_base = ffi::AVRational {
num: 1,
den: fps as i32,
};
(*par).hw_frames_ctx = frames_rgb.ref_clone();
let ret = ffi::av_buffersrc_parameters_set(src_ctx.as_mut_ptr(), par);
ffi::av_free(par as *mut _);
if ret < 0 {
bail!("av_buffersrc_parameters_set failed: {}", ff_err(ret));
}
}
// scale_vaapi: hardware scaling and colourspace conversion (keeps original dimensions).
let mut scale_ctx = graph.add(
&scale_vaapi,
"scale",
&format!("{width}:{height}:format=nv12"),
)?;
// SAFETY: scale_vaapi needs hw_device_ctx for VAAPI device access.
unsafe {
(*scale_ctx.as_mut_ptr()).hw_device_ctx = hw_dev.ref_clone();
}
let mut sink_ctx = graph.add(&buffersink, "out", "")?;
src_ctx.link(0, &mut scale_ctx, 0);
match transform {
Transform::Normal => {
scale_ctx.link(0, &mut sink_ctx, 0);
}
other => {
let transpose = ff::filter::find("transpose_vaapi")
.ok_or_else(|| anyhow::anyhow!("filter 'transpose_vaapi' not found"))?;
let dir_val = match other {
Transform::Normal90 => "1",
Transform::Normal180 => "4",
Transform::Normal270 => "2",
Transform::Flipped => "5",
Transform::Flipped90 => "3",
Transform::Flipped180 => "6",
Transform::Flipped270 => "0",
Transform::Normal => unreachable!(),
};
let mut trans_ctx = graph.add(&transpose, "transpose", &format!("dir={dir_val}"))?;
// SAFETY: trans_ctx is a live transpose_vaapi filter context;
// scale_vaapi/transpose_vaapi keep a ref-counted device context.
unsafe {
(*trans_ctx.as_mut_ptr()).hw_device_ctx = hw_dev.ref_clone();
}
scale_ctx.link(0, &mut trans_ctx, 0);
trans_ctx.link(0, &mut sink_ctx, 0);
}
}
graph
.validate()
.map_err(|e| anyhow::anyhow!("Filter graph validation failed: {e}"))?;
Ok(graph)
}
+285
View File
@@ -0,0 +1,285 @@
use std::path::Path;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use ffmpeg_next::packet::Mut as _;
use crate::transform::Transform;
use super::{
ff_err, filter::build_filter_graph, hardware_encoder, hardware_muxer, AvHwDevCtx, AvHwFrameCtx,
EncodeStages,
};
pub struct EncState {
enc_video: ff::codec::encoder::video::Video,
frames_rgb: AvHwFrameCtx,
video_filter: ff::filter::Graph,
// Root AVHWDeviceContext, kept for ownership. Each consumer (encoder,
// filter graph, frames ctx) already holds its own ref_clone(); this
// field is never read after `new()` but must outlive those clones.
#[allow(dead_code)]
hw_device_ctx: AvHwDevCtx,
octx: ff::format::context::Output,
starting_timestamp: Option<i64>,
frames_written: bool,
}
// SAFETY: EncState is moved to exactly one encode worker thread and all Rust
// methods take &mut self, so there is no concurrent *Rust-side* access.
// FFmpeg-internal codec threads may touch hw_device_ctx / frames_rgb through
// the encoder context if frame/slice threading is enabled; this is sound
// because AVHWDeviceContext and AVHWFramesContext are designed for such
// sharing (atomic refcounts, thread-safe pool, libva VADisplay thread safety).
// This impl only lifts auto-Send inference through raw pointers inside the
// ffmpeg-next wrappers; it does not introduce new sharing. Do NOT add fields
// that create shared mutable state across threads without re-auditing.
unsafe impl Send for EncState {}
impl EncState {
#[allow(clippy::too_many_arguments)]
pub fn new(
drm_device: &Path,
output_path: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
bitrate: u64,
gop_size: u32,
fps: u32,
transform: Transform,
existing_hw_ctx: Option<AvHwDevCtx>,
) -> Result<Self> {
tracing::info!(
"EncState::new: {width}x{height} enc={enc_width}x{enc_height} transform={transform:?}"
);
let hw_device_ctx = match existing_hw_ctx {
Some(ctx) => ctx,
None => AvHwDevCtx::new_vaapi(drm_device)?,
};
let frames_rgb =
AvHwFrameCtx::for_capture(&hw_device_ctx, width, height, ff::format::Pixel::BGRA)?;
let mut video_filter =
build_filter_graph(&hw_device_ctx, &frames_rgb, width, height, fps, transform)?;
let mut sink_ctx = video_filter
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
// SAFETY: sink_ctx is a live buffersink; the returned hw_frames_ctx is
// borrowed, so av_buffer_ref creates an owned reference.
let sink_hw_frames = unsafe {
let raw = ffi::av_buffersink_get_hw_frames_ctx(sink_ctx.as_mut_ptr());
if raw.is_null() {
bail!("buffersink has no hw_frames_ctx - filter graph may not be configured for hardware output");
}
let hw_ref = ffi::av_buffer_ref(raw);
if hw_ref.is_null() {
bail!("av_buffer_ref failed for buffersink hw_frames_ctx - likely out of memory");
}
hw_ref
};
// SAFETY: sink_hw_frames is an owned AVBufferRef to an AVHWFramesContext
// returned by the validated filter graph.
unsafe {
let fc = (*sink_hw_frames).data as *mut ffi::AVHWFramesContext;
let actual_w = (*fc).width as u32;
let actual_h = (*fc).height as u32;
if actual_w != enc_width || actual_h != enc_height {
tracing::warn!(
"Filter output dimensions {actual_w}x{actual_h} differ from encoder dimensions {enc_width}x{enc_height}"
);
}
}
let enc_video = hardware_encoder::open_h264_vaapi_encoder(
&hw_device_ctx,
sink_hw_frames,
enc_width,
enc_height,
bitrate,
gop_size,
fps,
)?;
let octx = hardware_muxer::create_output_context(output_path, &enc_video)?;
Ok(Self {
enc_video,
frames_rgb,
video_filter,
hw_device_ctx,
octx,
starting_timestamp: None,
frames_written: false,
})
}
pub fn frames_rgb(&self) -> &AvHwFrameCtx {
&self.frames_rgb
}
pub fn encode_frame(&mut self, hw_frame: &ff::frame::Video) -> Result<EncodeStages> {
let mut filter_src_ctx = self
.video_filter
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source();
let mut filter_sink_ctx = self
.video_filter
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink();
// Scale stage = filter graph push + pull (scale_vaapi for resolution
// change + format conversion to NV12). Timed separately from the
// actual avcodec_send_frame so the per-stage stats answer "where is
// latency?" honestly. See Oracle audit 2026-06-28 step 4.
let scale_start = Instant::now();
filter_src
.add(hw_frame)
.map_err(|e| anyhow::anyhow!("Filter source add failed: {e}"))?;
let mut scale_us = 0u64;
let mut encode_us = 0u64;
loop {
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => {
if filtered.pts().is_none() {
filtered.set_pts(hw_frame.pts());
}
}
Err(ff::Error::Other { errno }) if errno == ffi::EAGAIN => break,
Err(e) => bail!("Filter sink get frame failed: {e}"),
}
// First successful pull closes the scale-stage measurement; later
// pulls (rare extras) roll into encode time.
if scale_us == 0 {
scale_us = scale_start.elapsed().as_micros() as u64;
}
let pts = filtered.pts().unwrap_or(0);
if self.starting_timestamp.is_none() {
self.starting_timestamp = Some(pts);
}
let start_ts = self.starting_timestamp.unwrap();
let encode_start = Instant::now();
// SAFETY: avcodec_send_frame sends a valid NV12 VAAPI surface to the encoder.
let ret =
unsafe { ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), filtered.as_ptr()) };
if ret < 0 {
bail!("avcodec_send_frame failed: {}", ff_err(ret));
}
self.drain_encoder(start_ts)?;
encode_us += encode_start.elapsed().as_micros() as u64;
}
Ok(EncodeStages {
scale_us,
// HW path stays on GPU - no CPU readback, transfer is N/A.
transfer_us: 0,
encode_us,
})
}
pub fn flush(&mut self) -> Result<()> {
let mut filter_src_ctx = self
.video_filter
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source();
if let Err(e) = filter_src.flush() {
tracing::debug!("filter source flush error: {e}");
}
let mut filter_sink_ctx = self
.video_filter
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink();
loop {
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => {
let start_ts = self.starting_timestamp.unwrap_or(0);
// SAFETY: filtered is a valid VAAPI frame drained from the
// filter graph; enc_video is an opened encoder.
let ret = unsafe {
ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), filtered.as_ptr())
};
if ret < 0 {
bail!("avcodec_send_frame failed during flush: {}", ff_err(ret));
}
self.drain_encoder(start_ts)?;
}
Err(_) => break,
}
}
// SAFETY: Sending null frame signals end of stream to encoder.
unsafe {
ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), std::ptr::null());
}
let start_ts = self.starting_timestamp.unwrap_or(0);
self.drain_encoder(start_ts)?;
if self.frames_written {
self.octx
.write_trailer()
.map_err(|e| anyhow::anyhow!("Failed to write trailer: {e}"))?;
}
Ok(())
}
fn drain_encoder(&mut self, start_ts: i64) -> Result<()> {
loop {
let mut pkt = ff::Packet::empty();
// SAFETY: avcodec_receive_packet retrieves an encoded packet.
let ret = unsafe {
ffi::avcodec_receive_packet(self.enc_video.as_mut_ptr(), pkt.as_mut_ptr())
};
if ret < 0 {
if ret == ffi::AVERROR(ffi::EAGAIN) || ret == ffi::AVERROR_EOF {
break;
}
bail!("avcodec_receive_packet failed: {}", ff_err(ret));
}
let enc_tb = self.enc_video.time_base();
// SAFETY: octx was created with stream 0 during muxer setup; streams
// is non-null and stream 0 remains owned by the format context.
let stream_tb = unsafe {
let fmt = *self.octx.as_ptr();
if fmt.nb_streams == 0 || fmt.streams.is_null() {
bail!("no streams in output context");
}
let st = *fmt.streams.add(0);
ff::Rational::from((*st).time_base)
};
pkt.rescale_ts(enc_tb, stream_tb);
if let Some(pts) = pkt.pts() {
pkt.set_pts(Some(pts - start_ts));
}
if let Some(dts) = pkt.dts() {
pkt.set_dts(Some(dts - start_ts));
}
pkt.set_stream(0);
pkt.write_interleaved(&mut self.octx)
.map_err(|e| anyhow::anyhow!("Failed to write packet: {e}"))?;
self.frames_written = true;
}
Ok(())
}
}
+83
View File
@@ -0,0 +1,83 @@
use std::ffi::CString;
use anyhow::Result;
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::{ff_err, AvHwDevCtx};
#[allow(clippy::too_many_arguments)]
pub(super) fn open_h264_vaapi_encoder(
hw_device_ctx: &AvHwDevCtx,
sink_hw_frames: *mut ffi::AVBufferRef,
enc_width: u32,
enc_height: u32,
bitrate: u64,
gop_size: u32,
fps: u32,
) -> Result<ff::codec::encoder::video::Video> {
let codec = ff::encoder::find_by_name("h264_vaapi")
.ok_or_else(|| anyhow::anyhow!("h264_vaapi encoder not found"))?;
let mut enc = {
let ctx = ff::codec::Context::new_with_codec(codec);
ctx.encoder().video()?
};
enc.set_width(enc_width);
enc.set_height(enc_height);
enc.set_format(ff::format::Pixel::VAAPI);
enc.set_bit_rate(bitrate as usize);
enc.set_gop(gop_size);
enc.set_time_base(ff::Rational::new(1, fps as i32));
enc.set_max_b_frames(0);
// VBV rate limiting: caps IDR burst size for WebRTC. Without this a 4K
// scene change can produce a 256KB keyframe that overflows the UDP send
// buffer. bufsize=bitrate/4 is about 250ms of video at the target bitrate.
// SAFETY: enc.as_mut_ptr() is a valid AVCodecContext for the not-yet-opened
// encoder. rc_max_rate and rc_buffer_size are plain integer fields; assigning
// i64/i32 values is a simple struct-field write on a properly aligned pointer.
unsafe {
let ctx_ptr = enc.as_mut_ptr();
(*ctx_ptr).rc_max_rate = bitrate as i64;
(*ctx_ptr).rc_buffer_size = (bitrate / 4) as i32;
}
// SAFETY: AV_CODEC_FLAG_GLOBAL_HEADER must be set BEFORE opening the encoder.
// It triggers SPS/PPS extradata generation needed by the muxer for
// Annex B to AVCC conversion.
unsafe {
(*enc.as_mut_ptr()).flags |= ffi::AV_CODEC_FLAG_GLOBAL_HEADER as i32;
}
// SAFETY: Assign hw device and frames ctx to the encoder.
unsafe {
(*enc.as_mut_ptr()).hw_device_ctx = hw_device_ctx.ref_clone();
(*enc.as_mut_ptr()).hw_frames_ctx = sink_hw_frames;
}
// SAFETY: Set repeat_pps=1 on the encoder so PPS is inserted in every encoded frame.
// This ensures decoders can start decoding from any frame (important for WebRTC).
// repeat_pps is only available in FFmpeg 7.0+ (not in 6.x). On older
// FFmpeg, IDR frames carry SPS by default; PPS repetition depends on the driver.
// For SPS repetition: IDR frames carry SPS by default, controlled by gop_size/idr_interval.
{
let key = CString::new("repeat_pps").unwrap();
let val = CString::new("1").unwrap();
// SAFETY: enc is a valid AVCodecContext for the not-yet-opened encoder;
// priv_data is the codec's private options struct. key/val are NUL-terminated
// CString that live across the call. av_opt_set is FFmpeg's standard
// option-setter. Failure is non-fatal (returns < 0 on older FFmpeg).
let ret = unsafe {
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0)
};
if ret < 0 {
tracing::warn!("av_opt_set repeat_pps failed ({}), likely FFmpeg < 7.0; continuing without per-frame PPS", ff_err(ret));
}
}
let opened = enc
.open()
.map_err(|e| anyhow::anyhow!("Failed to open h264_vaapi encoder: {e}"))?;
Ok(opened.0)
}
+92
View File
@@ -0,0 +1,92 @@
use std::ffi::CString;
use std::path::Path;
use std::ptr;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::ff_err;
pub(super) fn create_output_context(
output_path: &Path,
enc_video: &ff::codec::encoder::video::Video,
) -> Result<ff::format::context::Output> {
let output_cstr = CString::new(output_path.to_str().unwrap())?;
let mut fmt_ctx_ptr: *mut ffi::AVFormatContext = ptr::null_mut();
// SAFETY: avformat_alloc_output_context2 creates format context from
// the file extension. Does NOT open the file.
let ret = unsafe {
ffi::avformat_alloc_output_context2(
&mut fmt_ctx_ptr,
ptr::null_mut(),
ptr::null(),
output_cstr.as_ptr(),
)
};
if ret < 0 || fmt_ctx_ptr.is_null() {
bail!("Failed to allocate output format context: {}", ff_err(ret));
}
// SAFETY: enc_video is a valid AVCodecContext pointer; codec_id is a plain
// i32 enum discriminant read from it. fmt_ctx_ptr is a valid AVFormatContext
// allocated above; oformat is a const pointer field read from it.
// avformat_query_codec checks codec+format compatibility; both pointers are
// valid and FF_COMPLIANCE_NORMAL is a constant. All three reads happen in one
// block so a single SAFETY rationale covers them.
let compat = unsafe {
let codec_id = (*enc_video.as_ptr()).codec_id;
let oformat = (*fmt_ctx_ptr).oformat;
ffi::avformat_query_codec(oformat, codec_id, ffi::FF_COMPLIANCE_NORMAL)
};
if compat < 0 {
bail!("H.264 codec not supported by output container format");
}
// SAFETY: avformat_new_stream creates a new stream in the format context.
let stream_ptr = unsafe { ffi::avformat_new_stream(fmt_ctx_ptr, ptr::null()) };
if stream_ptr.is_null() {
bail!("Failed to create new stream in output context");
}
// SAFETY: avcodec_parameters_from_context copies encoder params + extradata.
let ret =
unsafe { ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr()) };
if ret < 0 {
bail!(
"Failed to copy encoder parameters to stream: {}",
ff_err(ret)
);
}
// SAFETY: Copy encoder time_base to stream.
unsafe {
(*stream_ptr).time_base = (*enc_video.as_ptr()).time_base;
}
// SAFETY: avio_open opens the output file for writing.
let ret = unsafe {
ffi::avio_open(
&mut (*fmt_ctx_ptr).pb,
output_cstr.as_ptr(),
ffi::AVIO_FLAG_WRITE,
)
};
if ret < 0 {
bail!(
"Failed to open output file '{}': {}",
output_path.display(),
ff_err(ret)
);
}
// SAFETY: avformat_write_header writes the container header.
let ret = unsafe { ffi::avformat_write_header(fmt_ctx_ptr, ptr::null_mut()) };
if ret < 0 {
bail!("Failed to write output header: {}", ff_err(ret));
}
// SAFETY: We created fmt_ctx_ptr above and it's valid.
Ok(unsafe { ff::format::context::Output::wrap(fmt_ctx_ptr) })
}
+23
View File
@@ -0,0 +1,23 @@
const FNV1A_OFFSET_BASIS: u64 = 0xcbf29ce484222325;
const FNV1A_PRIME: u64 = 0x100000001b3;
const Y_PLANE_HASH_ROW_STEP: usize = 8;
pub(super) fn hash_sampled_y_plane(
y_data: &[u8],
width: usize,
height: usize,
stride: usize,
) -> u64 {
let mut hash = FNV1A_OFFSET_BASIS;
for row in (0..height).step_by(Y_PLANE_HASH_ROW_STEP) {
let row_start = row * stride;
let row_end = row_start + width;
for &byte in &y_data[row_start..row_end] {
hash ^= u64::from(byte);
hash = hash.wrapping_mul(FNV1A_PRIME);
}
}
hash
}
+263
View File
@@ -0,0 +1,263 @@
use std::path::Path;
use std::slice;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::filter::build_swenc_filter_graph;
use super::{ff_err, AvHwDevCtx, AvHwFrameCtx};
use super::{BitrateCommand, CpuNv12Frame, ResolutionChange};
pub struct SwEncImport {
hw_dev: AvHwDevCtx,
frames_rgb: AvHwFrameCtx,
filter_graph: ff::filter::Graph,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
resolution_rx: Option<crossbeam_channel::Receiver<BitrateCommand>>,
encoder_resolution_tx: Option<crossbeam_channel::Sender<ResolutionChange>>,
}
impl SwEncImport {
#[allow(clippy::too_many_arguments)]
pub fn new(
drm_device: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
) -> Result<Self> {
let hw_dev = AvHwDevCtx::new_vaapi(drm_device)?;
let frames_rgb =
AvHwFrameCtx::for_capture(&hw_dev, width, height, ff::format::Pixel::BGRA)?;
let filter_graph = build_swenc_filter_graph(
&hw_dev,
&frames_rgb,
width,
height,
enc_width,
enc_height,
fps,
)?;
Ok(Self {
hw_dev,
frames_rgb,
filter_graph,
width,
height,
enc_width,
enc_height,
fps,
resolution_rx: None,
encoder_resolution_tx: None,
})
}
#[allow(clippy::too_many_arguments)]
pub fn new_with_resolution_control(
drm_device: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
resolution_rx: crossbeam_channel::Receiver<BitrateCommand>,
encoder_resolution_tx: crossbeam_channel::Sender<ResolutionChange>,
) -> Result<Self> {
let mut this = Self::new(drm_device, width, height, enc_width, enc_height, fps)?;
this.resolution_rx = Some(resolution_rx);
this.encoder_resolution_tx = Some(encoder_resolution_tx);
Ok(this)
}
pub fn frames_rgb(&self) -> &AvHwFrameCtx {
let _ = self.hw_dev.as_ptr();
&self.frames_rgb
}
pub fn import_and_scale(&mut self, hw_frame: &ff::frame::Video) -> Result<CpuNv12Frame> {
self.poll_resolution_commands()?;
let mut filter_src_ctx = self
.filter_graph
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source();
let mut filter_sink_ctx = self
.filter_graph
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink();
filter_src
.add(hw_frame)
.map_err(|e| anyhow::anyhow!("software pipeline filter source add failed: {e}"))?;
let mut first = None;
let mut extra_count = 0usize;
loop {
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => {
if filtered.pts().is_none() {
filtered.set_pts(hw_frame.pts());
}
let cpu_frame = self.transfer_filtered_to_cpu(&filtered)?;
if first.is_none() {
first = Some(cpu_frame);
} else {
extra_count += 1;
}
}
Err(ff::Error::Other { errno }) if errno == ffi::EAGAIN => break,
Err(e) => bail!("software pipeline filter sink get frame failed: {e}"),
}
}
if extra_count > 0 {
tracing::warn!(
"software import filter produced {extra_count} extra frame(s); dropping extras"
);
}
first.ok_or_else(|| anyhow::anyhow!("software pipeline produced no scaled frame"))
}
pub fn flush_import(&mut self) -> Result<Vec<CpuNv12Frame>> {
let mut filter_src_ctx = self
.filter_graph
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source();
if let Err(e) = filter_src.flush() {
tracing::debug!("filter source flush error: {e}");
}
let mut filter_sink_ctx = self
.filter_graph
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink();
let mut frames = Vec::new();
loop {
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => frames.push(self.transfer_filtered_to_cpu(&filtered)?),
Err(_) => break,
}
}
Ok(frames)
}
fn poll_resolution_commands(&mut self) -> Result<()> {
let Some(rx) = self.resolution_rx.as_ref().cloned() else {
return Ok(());
};
let mut requested = None;
while let Ok(cmd) = rx.try_recv() {
match cmd {
BitrateCommand::UpdateResolution { width, height } => {
requested = Some((width & !1, height & !1));
}
BitrateCommand::UpdateBitrate { .. } => {}
BitrateCommand::ForceKeyframe => {}
}
}
let Some((width, height)) = requested else {
return Ok(());
};
if width == self.enc_width && height == self.enc_height {
return Ok(());
}
tracing::info!(
from = format_args!("{}x{}", self.enc_width, self.enc_height),
to = format_args!("{}x{}", width, height),
"rebuilding software import filter graph for resolution change"
);
let _ = self.flush_import();
self.filter_graph = build_swenc_filter_graph(
&self.hw_dev,
&self.frames_rgb,
self.width,
self.height,
width,
height,
self.fps,
)?;
self.enc_width = width;
self.enc_height = height;
if let Some(tx) = &self.encoder_resolution_tx {
tx.send(ResolutionChange { width, height })
.map_err(|_| anyhow::anyhow!("encoder resolution channel disconnected"))?;
}
Ok(())
}
fn transfer_filtered_to_cpu(&self, filtered: &ff::frame::Video) -> Result<CpuNv12Frame> {
// SAFETY: av_frame_alloc returns a newly allocated AVFrame or null,
// which is checked below.
let mut sw_nv12 = unsafe { ffi::av_frame_alloc() };
if sw_nv12.is_null() {
bail!("av_frame_alloc failed for NV12 transfer frame");
}
// SAFETY: sw_nv12 is an allocated destination frame; filtered is a valid VAAPI NV12
// surface produced by scale_vaapi at encoder dimensions.
let transfer_ret = unsafe { ffi::av_hwframe_transfer_data(sw_nv12, filtered.as_ptr(), 0) };
if transfer_ret < 0 {
// SAFETY: sw_nv12 was allocated above and has not been freed yet.
unsafe { ffi::av_frame_free(&mut sw_nv12) };
bail!(
"av_hwframe_transfer_data failed for GPU-downscaled frame: {}",
ff_err(transfer_ret)
);
}
// SAFETY: sw_nv12 was filled by av_hwframe_transfer_data. NV12 planes 0 and 1 are
// initialized for enc_width x enc_height; linesize values define each row's byte span.
let frame = unsafe {
let y_ptr = (*sw_nv12).data[0];
let uv_ptr = (*sw_nv12).data[1];
if y_ptr.is_null() || uv_ptr.is_null() {
ffi::av_frame_free(&mut sw_nv12);
bail!("NV12 transfer frame missing Y/UV plane data");
}
let y_stride = (*sw_nv12).linesize[0] as usize;
let uv_stride = (*sw_nv12).linesize[1] as usize;
if (*sw_nv12).width != self.enc_width as i32
|| (*sw_nv12).height != self.enc_height as i32
{
ffi::av_frame_free(&mut sw_nv12);
bail!("NV12 transfer frame has unexpected dimensions");
}
let y_len = y_stride * self.enc_height as usize;
let uv_len = uv_stride * (self.enc_height as usize / 2);
let y_data = slice::from_raw_parts(y_ptr, y_len).to_vec();
let uv_data = slice::from_raw_parts(uv_ptr, uv_len).to_vec();
let pts = filtered.pts().unwrap_or(0);
ffi::av_frame_free(&mut sw_nv12);
CpuNv12Frame {
y_data,
uv_data,
y_stride,
uv_stride,
pts,
capture_time: std::time::Instant::now(),
}
};
Ok(frame)
}
}
+258
View File
@@ -0,0 +1,258 @@
//! FFmpeg / VAAPI encoder wrappers.
//!
//! ## `Send` justification convention
//!
//! Several types in this module (`AvHwDevCtx`, `AvHwFrameCtx`, `EncState`,
//! `SwEncState`, `SwEncEncode`) carry raw FFmpeg pointers and therefore need
//! an explicit `unsafe impl Send`. The justification is always at the C-API
//! level, never at the Rust-borrow level:
//!
//! - `AVBufferRef` refcounts are `atomic_uint` (`libavutil/buffer.c`), so
//! `av_buffer_ref` / `av_buffer_unref` are safe to call concurrently.
//! - `AVHWDeviceContext` (VAAPI `VADisplay`) is designed by FFmpeg/libva to
//! be shared across codec and filter contexts, including FFmpeg-internal
//! codec worker threads.
//! - `AVHWFramesContext` allocates from an `AVBufferPool` whose get/put are
//! atomic; `av_hwframe_get_buffer` is safe to call concurrently on
//! distinct frames.
//! - `SwsContext`, `AVFilterGraph`, `AVFrame`, `AVCodecContext` are NOT
//! thread-safe for concurrent use, but are `Send`-sound under the
//! single-thread exclusive access invariant that the encode worker
//! enforces.
//!
//! **Anti-pattern**: justifying `Send` with "`&mut self` ensures exclusive
//! access". `Send` is about *moving ownership between threads*, not about
//! borrowing. The `&mut self` on Rust methods is API convenience and is not
//! the basis for soundness — refs cloned via `ref_clone()` routinely escape
//! to other threads / FFmpeg-internal workers, and that is fine because the
//! underlying C APIs are designed for it.
use std::path::Path;
use anyhow::Result;
use crate::transform::{transpose_if_transform_transposed, Transform};
mod device;
mod dmabuf;
mod encode;
mod encode_init;
mod encode_output;
mod filter;
mod hardware;
mod hardware_encoder;
mod hardware_muxer;
mod hash;
mod import;
mod software;
mod state;
mod types;
mod util;
pub use device::{AvHwDevCtx, AvHwFrameCtx};
pub use dmabuf::{import_dma_buf_to_vaapi, test_dma_buf_import};
pub use encode::{SwEncEncode, WEBRTC_RTP_CLOCK_HZ};
#[allow(unused_imports)]
pub use encode_output::FrameOutput;
pub use hardware::EncState;
#[cfg(test)]
use hash::hash_sampled_y_plane;
pub use import::SwEncImport;
pub use state::SwEncState;
pub use types::{
BitrateCommand, CpuNv12Frame, EncodeOutcome, EncodeStages, EncodedH264Frame, ResolutionChange,
SwEncodeTiming,
};
#[allow(unused_imports)]
pub use util::av_err_to_string;
pub(crate) use util::ff_err;
// ---------------------------------------------------------------------------
// Shared encoder creation (used by both wlr-screencopy and portal paths)
// ---------------------------------------------------------------------------
/// Create a fully configured encoder with VAAPI hardware acceleration.
///
/// Convenience wrapper around [`EncState::new`] that computes default values
/// for `bitrate` and `gop_size` when not provided, and handles encoder dimension
/// transposition for rotated/transformed outputs.
#[allow(clippy::too_many_arguments)]
pub fn create_encoder(
drm_device: &Path,
output_path: &Path,
width: u32,
height: u32,
fps: u32,
transform: Transform,
bitrate: Option<u64>,
gop_size: Option<u32>,
existing_hw_ctx: Option<AvHwDevCtx>,
) -> Result<EncState> {
let (enc_w, enc_h) = transpose_if_transform_transposed(transform, width as i32, height as i32);
let actual_bitrate =
bitrate.unwrap_or_else(|| 2 * (width as u64) * (height as u64) * (fps as u64) / 100);
let actual_gop_size = gop_size.unwrap_or(fps);
EncState::new(
drm_device,
output_path,
width,
height,
enc_w as u32,
enc_h as u32,
actual_bitrate,
actual_gop_size,
fps,
transform,
existing_hw_ctx,
)
}
#[cfg(test)]
mod tests {
use super::*;
// Centralizes the `stride * row` byte-offset pattern used by the Y-plane hash
// tests below, so clippy::erasing_op (row == 0) and clippy::identity_op (row == 1)
// both pass without sacrificing the row-index intent the tests are written around.
fn row_range(row: usize, stride: usize, width: usize) -> std::ops::Range<usize> {
let start = stride * row;
start..start + width
}
// ── Task 1: VBV x264opts formatting ──
#[test]
fn vbv_x264opts_format() {
let bitrate: u64 = 5_000_000;
// x264 expects kbit/s and kbit, not bps
let vbv_maxrate_kbps = bitrate / 1000;
let vbv_bufsize_kbps = (bitrate / 4) / 1000;
let opts = format!(
"repeat_headers=1:vbv-maxrate={vbv_maxrate_kbps}:vbv-bufsize={vbv_bufsize_kbps}"
);
assert_eq!(vbv_maxrate_kbps, 5000);
assert_eq!(vbv_bufsize_kbps, 1250);
assert!(opts.contains("vbv-maxrate=5000"));
assert!(opts.contains("vbv-bufsize=1250"));
}
#[test]
fn vbv_bufsize_is_quarter_of_maxrate() {
for bitrate in [1_000_000, 5_000_000, 10_000_000] {
// x264 expects kbit/s and kbit; both scaled by /1000, ratio preserved
let maxrate_kbps = bitrate / 1000;
let bufsize_kbps = (bitrate / 4) / 1000;
assert_eq!(
bufsize_kbps * 4,
maxrate_kbps,
"bufsize should be maxrate/4"
);
}
}
// ── Task 3: GOP formula ──
#[test]
fn webrtc_gop_formula() {
// Formula under test: GOP = max(fps * 2, 20). Hid behind a runtime lambda so
// clippy can't constant-fold the assertions into tautologies (which would
// silently strip the floor-case coverage for 5fps).
fn gop(fps: u32) -> u32 {
(fps * 2).max(20)
}
assert_eq!(gop(15), 30); // 15fps -> 30
assert_eq!(gop(30), 60); // 30fps -> 60
assert_eq!(gop(60), 120); // 60fps -> 120
assert_eq!(gop(5), 20); // 5fps -> 20 (floor)
}
#[test]
fn h264_level_values() {
// Level 4.0 supports up to 1080p@30fps (used for file muxer)
assert_eq!(40i32, 40);
// Level 4.2 supports up to 1440p@30fps (used for WebRTC low-latency encoder)
assert_eq!(42i32, 42);
}
// ── Task 4: Duplicate frame hash detection ──
#[test]
fn hash_sampled_y_plane_first_frame_consistent() {
let width = 64;
let height = 64;
let stride = 64;
let y_data = vec![0u8; stride * height];
let hash1 = hash_sampled_y_plane(&y_data, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data, width, height, stride);
assert_eq!(hash1, hash2, "same input should produce same hash");
}
#[test]
fn hash_sampled_y_plane_detects_different_frames() {
let width = 64;
let height = 64;
let stride = 64;
let y_data1 = vec![0u8; stride * height];
let y_data2 = vec![128u8; stride * height];
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_ne!(
hash1, hash2,
"different frame data should produce different hashes"
);
}
#[test]
fn hash_sampled_y_plane_samples_every_8th_row() {
// Changing a non-sampled row (e.g., row 1) should NOT change the hash
let width = 64;
let height = 64;
let stride = 64;
let y_data1 = vec![0u8; stride * height];
let mut y_data2 = vec![0u8; stride * height];
// Row 1 is NOT sampled (sampling is every 8th row: 0, 8, 16, ...)
y_data2[row_range(1, stride, width)].fill(255);
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_eq!(
hash1, hash2,
"non-sampled row change should not affect hash"
);
}
#[test]
fn hash_sampled_y_plane_sensitive_to_sampled_row() {
// Changing a sampled row (row 0) SHOULD change the hash
let width = 64;
let height = 64;
let stride = 64;
let y_data1 = vec![0u8; stride * height];
let mut y_data2 = vec![0u8; stride * height];
// Row 0 IS sampled (every 8th row starting from 0)
y_data2[row_range(0, stride, width)].fill(255);
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_ne!(
hash1, hash2,
"sampled row change should produce different hash"
);
}
#[test]
fn hash_sampled_y_plane_handles_stride_greater_than_width() {
// Stride can be larger than width due to alignment; unused padding should not affect hash
let width = 32;
let height = 16;
let stride = 64; // padded stride
let y_data1 = vec![0u8; stride * height];
let mut y_data2 = vec![0u8; stride * height];
// Fill the padding area (columns 32..63) of row 0 with garbage
y_data2[width..stride].fill(0xFF);
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_eq!(
hash1, hash2,
"padding bytes beyond width should not affect hash"
);
}
}
+268
View File
@@ -0,0 +1,268 @@
use std::ffi::CString;
use std::path::Path;
use std::ptr;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use super::ff_err;
pub(super) fn create_nv12_to_yuv420p_sws(width: u32, height: u32) -> Result<*mut ffi::SwsContext> {
// SAFETY: sws_getContext creates an owned scaler context for same-size NV12 -> YUV420P.
let ctx = unsafe {
ffi::sws_getContext(
width as i32,
height as i32,
ffi::AVPixelFormat::AV_PIX_FMT_NV12,
width as i32,
height as i32,
ffi::AVPixelFormat::AV_PIX_FMT_YUV420P,
2,
ptr::null_mut(),
ptr::null_mut(),
ptr::null_mut(),
)
};
if ctx.is_null() {
bail!("Failed to create NV12 -> YUV420P sws_scale context");
}
Ok(ctx)
}
pub(super) fn alloc_yuv420p_frame(width: u32, height: u32) -> Result<*mut ffi::AVFrame> {
// SAFETY: Allocate an AVFrame, configure format/dimensions, then allocate writable buffers.
unsafe {
let mut frame = ffi::av_frame_alloc();
if frame.is_null() {
bail!("av_frame_alloc failed");
}
(*frame).width = width as i32;
(*frame).height = height as i32;
(*frame).format = ffi::AVPixelFormat::AV_PIX_FMT_YUV420P as i32;
let ret = ffi::av_frame_get_buffer(frame, 0);
if ret < 0 {
ffi::av_frame_free(&mut frame);
bail!("av_frame_get_buffer failed: {}", ff_err(ret));
}
Ok(frame)
}
}
pub(super) fn create_software_h264_muxer(
output_path: &Path,
width: u32,
height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
) -> Result<(
ff::codec::encoder::video::Video,
ff::format::context::Output,
)> {
let output_cstr = CString::new(output_path.to_str().unwrap())?;
let codec = ff::encoder::find_by_name("libx264")
.or_else(|| ff::encoder::find_by_name("libopenh264"))
.ok_or_else(|| {
anyhow::anyhow!("No H.264 software encoder found (tried libx264, libopenh264)")
})?;
let codec_name = codec.name().to_string();
let mut enc = {
let ctx = ff::codec::Context::new_with_codec(codec);
ctx.encoder().video()?
};
enc.set_width(width);
enc.set_height(height);
enc.set_format(ff::format::Pixel::YUV420P);
enc.set_bit_rate(bitrate as usize);
enc.set_gop(gop_size);
enc.set_time_base(ff::Rational::new(1, fps as i32));
enc.set_max_b_frames(3);
// SAFETY: global headers are needed by MP4 and harmless for other common muxers.
unsafe {
(*enc.as_mut_ptr()).flags |= ffi::AV_CODEC_FLAG_GLOBAL_HEADER as i32;
}
if codec_name == "libx264" {
// SAFETY: priv_data and codec context belong to the unopened encoder;
// strings live for each av_opt_set call.
unsafe {
let key = CString::new("preset").unwrap();
let val = CString::new("fast").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("threads").unwrap();
let val = CString::new("6").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
(*enc.as_mut_ptr()).profile = ffi::AV_PROFILE_H264_HIGH;
// SAFETY: enc is a valid, initialized AVCodecContext from
// avcodec_alloc_context3. Setting level is a simple i32 field
// assignment on a properly aligned struct.
(*enc.as_mut_ptr()).level = 40; // H.264 Level 4.0 (up to 1080p@30)
}
}
let opened = enc
.open()
.map_err(|e| anyhow::anyhow!("Failed to open {codec_name} encoder: {e}"))?;
let enc_video = opened.0;
let use_null = output_path
.to_str()
.map(|s| s.contains("null"))
.unwrap_or(false);
let fmt_name = if use_null {
CString::new("null").unwrap()
} else {
CString::new("").unwrap()
};
let fmt_name_ptr = if use_null {
fmt_name.as_ptr()
} else {
ptr::null()
};
let mut fmt_ctx_ptr: *mut ffi::AVFormatContext = ptr::null_mut();
// SAFETY: fmt_ctx_ptr is initialized by FFmpeg; C strings live across the call.
let ret = unsafe {
ffi::avformat_alloc_output_context2(
&mut fmt_ctx_ptr,
ptr::null_mut(),
fmt_name_ptr,
output_cstr.as_ptr(),
)
};
if ret < 0 || fmt_ctx_ptr.is_null() {
bail!("Failed to allocate output format context: {}", ff_err(ret));
}
// SAFETY: fmt_ctx_ptr is valid; stream and codec parameters are owned by the format context.
let stream_ptr = unsafe { ffi::avformat_new_stream(fmt_ctx_ptr, ptr::null()) };
if stream_ptr.is_null() {
bail!("Failed to create output stream");
}
// SAFETY: stream_ptr and encoder context are valid; parameters are copied into stream.
let ret =
unsafe { ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr()) };
if ret < 0 {
bail!("Failed to copy codec parameters to stream: {}", ff_err(ret));
}
// SAFETY: stream_ptr is valid and writable during muxer setup.
unsafe {
(*stream_ptr).time_base = (*enc_video.as_ptr()).time_base;
}
// SAFETY: open an AVIO only for muxers that require files; null muxer advertises NOFILE.
unsafe {
if (*(*fmt_ctx_ptr).oformat).flags & ffi::AVFMT_NOFILE == 0 {
let ret = ffi::avio_open(
&mut (*fmt_ctx_ptr).pb,
output_cstr.as_ptr(),
ffi::AVIO_FLAG_WRITE,
);
if ret < 0 {
bail!(
"Failed to open output file '{}': {}",
output_path.display(),
ff_err(ret)
);
}
}
}
// SAFETY: fmt_ctx_ptr is fully configured.
let ret = unsafe { ffi::avformat_write_header(fmt_ctx_ptr, ptr::null_mut()) };
if ret < 0 {
bail!("Failed to write output header: {}", ff_err(ret));
}
// SAFETY: ownership of fmt_ctx_ptr transfers to ffmpeg-next Output wrapper.
let octx = unsafe { ff::format::context::Output::wrap(fmt_ctx_ptr) };
tracing::info!("Using software H.264 encoder: {codec_name}");
Ok((enc_video, octx))
}
pub(super) fn create_software_h264_encoder(
width: u32,
height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
) -> Result<ff::codec::encoder::video::Video> {
let codec = ff::encoder::find_by_name("libx264")
.or_else(|| ff::encoder::find_by_name("libopenh264"))
.ok_or_else(|| anyhow::anyhow!("No H.264 software encoder found"))?;
let codec_name = codec.name().to_string();
let mut enc = {
let ctx = ff::codec::Context::new_with_codec(codec);
ctx.encoder().video()?
};
enc.set_width(width);
enc.set_height(height);
enc.set_format(ff::format::Pixel::YUV420P);
enc.set_bit_rate(bitrate as usize);
enc.set_gop(gop_size);
// 90kHz media clock matches RTP directly. Eliminates 1/fps quantization
// that previously caused sequential RTP timestamps during 60fps capture,
// leading to 2x RTP time inflation and 10s+ browser jitter buffer growth.
// See issue #25.
enc.set_time_base(ff::Rational::new(1, 90_000));
// Explicit framerate is REQUIRED when time_base is not 1/fps, otherwise
// libx264 infers wrong fps from the 90kHz time_base and VBV rate control
// breaks. Per Oracle review round for #25.
enc.set_frame_rate(Some(ff::Rational::new(fps as i32, 1)));
enc.set_max_b_frames(0);
if codec_name == "libx264" {
// SAFETY: priv_data and codec context belong to the unopened encoder;
// each CString lives for the duration of its av_opt_set call.
unsafe {
let key = CString::new("preset").unwrap();
let val = CString::new("veryfast").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("tune").unwrap();
let val = CString::new("zerolatency").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("threads").unwrap();
let val = CString::new("6").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
// High profile via AVCodecContext.profile (not x264opts - x264 rejects it there).
// High enables CABAC + 8x8dct automatically.
(*enc.as_mut_ptr()).profile = ffi::AV_PROFILE_H264_HIGH;
// SAFETY: enc is a valid, initialized AVCodecContext from
// avcodec_alloc_context3. Setting level is a simple i32 field
// assignment on a properly aligned struct.
(*enc.as_mut_ptr()).level = 42; // H.264 Level 4.2 (up to 1440p@30)
// SAFETY: priv_data belongs to the unopened libx264 encoder context.
// `forced-idr` is an FFmpeg-level private option (not x264-native),
// so it must be set via av_opt_set, NOT via the x264opts string.
// With forced-idr=1, setting AV_PICTURE_TYPE_I on an input frame
// produces a true IDR NALU with inline SPS/PPS (repeat_headers=1).
let key = CString::new("forced-idr").unwrap();
let val = CString::new("1").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("x264opts").unwrap();
// x264's vbv-maxrate unit is kbit/s and vbv-bufsize is kbit (NOT bps).
// Confirmed via x264 source encoder/ratecontrol.c:658-661 which multiplies
// these values by 1000 to convert kbit -> bit at use site. Passing bps makes
// VBV effectively unbounded (5.5 Mbps becomes 5.5 Gbps, clipped to 2 Gbps).
// See https://github.com/mirror/x264/blob/c24e06c2e184345ceb33eb20a15d1024d9fd3497/encoder/ratecontrol.c#L658-L661
let vbv_maxrate_kbps = bitrate / 1000;
let vbv_bufsize_kbps = (bitrate / 4) / 1000;
let val = CString::new(format!(
"repeat_headers=1:vbv-maxrate={vbv_maxrate_kbps}:vbv-bufsize={vbv_bufsize_kbps}"
))
.unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
}
}
let opened = enc
.open()
.map_err(|e| anyhow::anyhow!("Failed to open {codec_name} encoder: {e}"))?;
tracing::info!("WebRTC encoder: {codec_name} {width}x{height} @ {fps}fps {bitrate}bps (profile High, preset veryfast)");
Ok(opened.0)
}
+111
View File
@@ -0,0 +1,111 @@
use std::path::Path;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use std::time::Instant;
use anyhow::Result;
use ffmpeg_next as ff;
use super::{AvHwFrameCtx, EncodeStages, EncodedH264Frame, SwEncEncode, SwEncImport};
pub struct SwEncState {
import: SwEncImport,
encode: SwEncEncode,
}
// SAFETY: SwEncState is moved to a single encode thread and accessed only there.
// All FFmpeg handles (SwsContext, AVFrame, AVCodecContext) inside SwEncImport /
// SwEncEncode are non-thread-safe but Send-sound under exclusive access.
// Existing sync callers move it across threads only with external serialization.
unsafe impl Send for SwEncState {}
impl SwEncState {
#[allow(clippy::too_many_arguments)]
pub fn new(
drm_device: &Path,
output_path: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
) -> Result<Self> {
tracing::info!(
"SwEncState::new: GPU downscale {width}x{height} BGRA -> {enc_width}x{enc_height} NV12, software H.264"
);
let import = SwEncImport::new(drm_device, width, height, enc_width, enc_height, fps)?;
let encode =
SwEncEncode::new_muxer(output_path, enc_width, enc_height, fps, bitrate, gop_size)?;
Ok(Self { import, encode })
}
#[allow(clippy::too_many_arguments)]
pub fn new_webrtc(
drm_device: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
tx: crossbeam_channel::Sender<EncodedH264Frame>,
webrtc_paused: Arc<AtomicBool>,
) -> Result<Self> {
tracing::info!(
"SwEncState::new_webrtc: GPU downscale {width}x{height} BGRA -> {enc_width}x{enc_height} NV12, software H.264 -> WebRTC"
);
let import = SwEncImport::new(drm_device, width, height, enc_width, enc_height, fps)?;
let (dummy_tx, bitrate_rx) = crossbeam_channel::bounded(1);
drop(dummy_tx);
let (dummy_resolution_tx, resolution_rx) = crossbeam_channel::bounded(1);
drop(dummy_resolution_tx);
let encode = SwEncEncode::new_webrtc(
enc_width,
enc_height,
fps,
bitrate,
gop_size,
tx,
webrtc_paused,
bitrate_rx,
resolution_rx,
)?;
Ok(Self { import, encode })
}
pub fn frames_rgb(&self) -> &AvHwFrameCtx {
self.import.frames_rgb()
}
pub fn encode_frame(&mut self, hw_frame: &ff::frame::Video) -> Result<EncodeStages> {
// SW path: import_and_scale bundles GPU filter graph (scale) + GPU->CPU
// readback (transfer) into one call. Timing them separately requires
// extending import_and_scale's signature; for now both roll into
// scale_us and transfer_us stays 0 with this comment as the honest
// statement. Oracle audit 2026-06-28 step 4.
let scale_start = Instant::now();
let cpu_frame = self.import.import_and_scale(hw_frame)?;
let scale_us = scale_start.elapsed().as_micros() as u64;
let encode_start = Instant::now();
self.encode.encode_cpu_frame(&cpu_frame)?;
let encode_us = encode_start.elapsed().as_micros() as u64;
Ok(EncodeStages {
scale_us,
transfer_us: 0,
encode_us,
})
}
pub fn flush(&mut self) -> Result<()> {
for frame in self.import.flush_import()? {
self.encode.encode_cpu_frame(&frame)?;
}
self.encode.flush()?;
self.encode.write_trailer_if_needed()
}
}
+88
View File
@@ -0,0 +1,88 @@
/// Commands sent from the WebRTC thread to the SW encoder when the
/// bandwidth estimate changes significantly.
pub enum BitrateCommand {
UpdateBitrate {
target_bps: u64,
},
UpdateResolution {
width: u32,
height: u32,
},
/// Force the next encoded frame to be an IDR. Sent by the WebRTC thread
/// in response to str0m `Event::KeyframeRequest` or a resolution change.
ForceKeyframe,
}
#[derive(Clone, Copy, Debug)]
pub struct ResolutionChange {
pub width: u32,
pub height: u32,
}
/// Per-frame timing snapshot for the software encoder, consumed by the stats
/// thread. `sws_us` measures NV12->YUV420P conversion, `encode_us` measures
/// `avcodec_send_frame` + drain, and `output_bytes` counts encoded bytes
/// produced by libavcodec (even if downstream delivery later drops them).
#[derive(Default, Clone, Copy, Debug)]
pub struct SwEncodeTiming {
pub sws_us: u64,
pub encode_us: u64,
pub output_bytes: usize,
}
/// Outcome of a single `encode_cpu_frame` call. Used by the encode thread
/// to decide whether to report timing stats (only real encodes tick encoded_fps).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum EncodeOutcome {
/// Frame was actually encoded and produced output bytes.
Encoded,
/// Frame was dropped because WebRTC is paused (no client connected).
SkippedPaused,
/// Frame was dropped because the encoder is in disconnected state.
SkippedDisconnected,
/// Frame was dropped because its Y-plane hash matched the previous frame.
SkippedDuplicate,
}
/// Per-stage timing breakdown for one encode cycle on the hardware path.
/// Returned by `EncState::encode_frame` so callers can fold the numbers
/// into `crate::stats::FrameTimings`. `transfer_us` is always 0 on the HW
/// path because the frame stays on the GPU; the SW path's struct (if added
/// later) would carry a real readback measurement.
#[derive(Debug, Default, Clone, Copy)]
pub struct EncodeStages {
pub scale_us: u64,
pub transfer_us: u64,
pub encode_us: u64,
}
/// Encoded H.264 frame with timing metadata for WebRTC output.
///
/// MP4 file output (FrameOutput::Muxer) does NOT use this - it writes via
/// avformat which preserves PTS internally. WebRTC output (FrameOutput::Channel)
/// requires explicit PTS propagation so RTP timestamps reflect real capture time.
/// Without this, WebRTC clients' jitter buffers grow to seconds under
/// damage-driven variable frame rate. See issue #24.
#[derive(Debug)]
pub struct EncodedH264Frame {
/// H.264 NAL byte stream (Annex B or AVCC depending on encoder configuration)
pub data: Vec<u8>,
/// PTS in encoder time_base units (1/fps seconds), normalized so first frame = 0.
/// Derived from real capture time, NOT frame counter.
pub pts_ticks: i64,
/// Wall-clock capture time, propagated from CpuNv12Frame for frame_age stat.
pub capture_time: std::time::Instant,
}
/// Owned CPU NV12 frame data for cross-thread transfer.
/// Produced by main thread (VAAPI import + GPU scale + transfer), consumed by encode thread.
pub struct CpuNv12Frame {
pub y_data: Vec<u8>,
pub uv_data: Vec<u8>,
pub y_stride: usize,
pub uv_stride: usize,
pub pts: i64,
/// Wall-clock time when this frame was captured (PipeWire delivery).
/// Used for frame_age stat: time from capture to WebRTC send.
pub capture_time: std::time::Instant,
}
+20
View File
@@ -0,0 +1,20 @@
use ffmpeg_next::ffi;
/// Convert an FFmpeg error code to a human-readable string.
pub fn av_err_to_string(err: i32) -> String {
let mut buf = vec![0u8; 128];
// SAFETY: buf points to 128 writable bytes and lives for the duration of
// av_strerror.
unsafe {
ffi::av_strerror(err, buf.as_mut_ptr() as *mut i8, buf.len());
}
String::from_utf8_lossy(&buf)
.trim_end_matches('\0')
.to_string()
}
/// Format an FFmpeg error code with both numeric value and description.
/// Example output: "error -22 (Invalid argument)"
pub(crate) fn ff_err(ret: i32) -> String {
format!("error {ret} ({})", av_err_to_string(ret))
}
+17 -19
View File
@@ -113,25 +113,23 @@ fn check_portal_available() -> bool {
// The most likely operation to hang — requires actual Portal-side work.
// 最可能卡住的操作,需要 Portal 端实际处理。
let version = match tokio::time::timeout(
PORTAL_DBUS_TIMEOUT,
inner.get_property::<u32>("version"),
)
.await
{
Ok(Ok(version)) => {
tracing::info!("Portal ScreenCast available (version: {version})");
true
}
Ok(Err(e)) => {
tracing::info!("Portal ScreenCast version query failed: {e}");
false
}
Err(_) => {
log_portal_unresponsive("querying ScreenCast version");
false
}
};
let version =
match tokio::time::timeout(PORTAL_DBUS_TIMEOUT, inner.get_property::<u32>("version"))
.await
{
Ok(Ok(version)) => {
tracing::info!("Portal ScreenCast available (version: {version})");
true
}
Ok(Err(e)) => {
tracing::info!("Portal ScreenCast version query failed: {e}");
false
}
Err(_) => {
log_portal_unresponsive("querying ScreenCast version");
false
}
};
version
})
}
+76
View File
@@ -0,0 +1,76 @@
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use ffmpeg_next::packet::Mut;
use wl_webrtc::cap_portal::{CapPortal, PwCtrlEvent};
pub fn receive_first_frame(cap: &CapPortal) -> Result<wl_webrtc::cap_portal::PwDmaBufFrame> {
// Drain-and-wait loop that mirrors production's repeated-poll semantics
// (state_portal.rs::poll_and_encode driven by main.rs's outer loop), but with
// a single bounded 10s total deadline appropriate for a bench tool. Unlike a
// single 10s blocking wait, this loop actually iterates: each turn drains ALL
// pending control events (the ctrl channel is bounded to 8 — a single
// if-let would silently miss backlog) and then waits a short slice for a
// frame, so StreamEnded/Error arriving mid-wait are observed within ~200ms.
const TOTAL_DEADLINE: std::time::Duration = std::time::Duration::from_secs(10);
const WAIT_SLICE: std::time::Duration = std::time::Duration::from_millis(200);
let deadline = Instant::now() + TOTAL_DEADLINE;
loop {
while let Ok(ctrl) = cap.event_receiver().try_recv() {
match ctrl {
PwCtrlEvent::StreamEnded => bail!("PipeWire stream ended before first frame"),
PwCtrlEvent::FormatChanged { .. } => {}
PwCtrlEvent::Error(e) => bail!("PipeWire error: {e}"),
}
}
let remaining = match deadline.checked_duration_since(Instant::now()) {
Some(r) if !r.is_zero() => r,
_ => bail!("Timeout waiting for first frame (10s)"),
};
let slice = remaining.min(WAIT_SLICE);
match cap.frame_receiver().recv_timeout(slice) {
Ok(frame) => return Ok(frame),
Err(crossbeam_channel::RecvTimeoutError::Timeout) => continue,
Err(crossbeam_channel::RecvTimeoutError::Disconnected) => {
bail!("PipeWire frame channel disconnected");
}
}
}
}
pub fn drain_encoder(
enc_video: &mut ff::encoder::video::Video,
octx: &mut ff::format::context::Output,
) -> Result<()> {
loop {
let mut pkt = ff::Packet::empty();
// SAFETY: enc_video is the opened encoder; pkt is an empty Packet whose
// inner AVPacket pointer is valid. avcodec_receive_packet fills pkt with
// the next encoded packet, or returns EAGAIN/EOF when drained.
let ret = unsafe { ffi::avcodec_receive_packet(enc_video.as_mut_ptr(), pkt.as_mut_ptr()) };
if ret < 0 {
if ret == ffi::AVERROR(ffi::EAGAIN) || ret == ffi::AVERROR_EOF {
break;
}
eprintln!("avcodec_receive_packet failed: {ret}");
break;
}
let enc_tb = enc_video.time_base();
// SAFETY: octx.as_ptr() is a valid AVFormatContext; streams is a NULL-terminated
// array of AVStream*. We index [0] which exists because we created exactly one
// stream in setup. Reading time_base is a plain AVRational field access.
let stream_tb = unsafe {
let streams = (*octx.as_ptr()).streams;
let st = *streams.add(0);
ff::Rational::from((*st).time_base)
};
pkt.rescale_ts(enc_tb, stream_tb);
pkt.set_stream(0);
pkt.write_interleaved(octx)
.map_err(|e| anyhow::anyhow!("write packet failed: {e}"))?;
}
Ok(())
}
@@ -15,75 +15,16 @@ use clap::Parser;
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use ffmpeg_next::packet::Mut;
use wl_webrtc::args::Args;
use wl_webrtc::cap_portal::{CapPortal, PwCtrlEvent};
#[derive(Parser, Debug)]
#[command(
name = "sw_encode_bench",
about = "Software encoding pipeline benchmark"
)]
struct BenchArgs {
#[arg(short, long)]
output: String,
#[path = "../common/mod.rs"]
mod common;
#[arg(long, default_value_t = 120)]
frames: u32,
mod stats;
#[arg(long, default_value_t = 2560)]
enc_width: u32,
#[arg(long, default_value_t = 1440)]
enc_height: u32,
}
#[derive(Default)]
struct FrameStats {
mmap_us: Vec<u64>,
scale_us: Vec<u64>,
encode_us: Vec<u64>,
total_us: Vec<u64>,
mmap_failures: u32,
}
impl FrameStats {
fn avg_ms(data: &[u64]) -> f64 {
if data.is_empty() {
return 0.0;
}
data.iter().sum::<u64>() as f64 / data.len() as f64 / 1000.0
}
}
fn pix_fmt(p: ff::format::Pixel) -> ffi::AVPixelFormat {
Into::<ffi::AVPixelFormat>::into(p)
}
fn receive_first_frame(cap: &CapPortal) -> Result<wl_webrtc::cap_portal::PwDmaBufFrame> {
loop {
if let Ok(ctrl) = cap.event_receiver().try_recv() {
match ctrl {
PwCtrlEvent::StreamEnded => bail!("PipeWire stream ended before first frame"),
PwCtrlEvent::FormatChanged { .. } => {}
PwCtrlEvent::Error(e) => bail!("PipeWire error: {e}"),
}
}
match cap
.frame_receiver()
.recv_timeout(std::time::Duration::from_secs(10))
{
Ok(frame) => return Ok(frame),
Err(crossbeam_channel::RecvTimeoutError::Timeout) => {
bail!("Timeout waiting for first frame (10s)");
}
Err(crossbeam_channel::RecvTimeoutError::Disconnected) => {
bail!("PipeWire frame channel disconnected");
}
}
}
}
use stats::{pix_fmt, BenchArgs, FrameStats};
fn main() -> Result<()> {
let bench_args = BenchArgs::parse();
@@ -123,7 +64,7 @@ fn main() -> Result<()> {
println!("[1/4] Portal connected, PipeWire stream active\n");
println!("[2/4] Waiting for first frame from PipeWire...");
let first_frame = receive_first_frame(&cap)?;
let first_frame = common::receive_first_frame(&cap)?;
let src_width = first_frame.width;
let src_height = first_frame.height;
@@ -142,6 +83,9 @@ fn main() -> Result<()> {
println!("[3/4] Testing mmap on DMA-BUF...");
let mmap_size = (src_stride as usize) * (src_height as usize);
// SAFETY: first_frame.fd is an open DMA-BUF; offset/size come from PipeWire's
// negotiated format. PROT_READ+MAP_SHARED is the standard read-only DMA-BUF
// mapping. Returns MAP_FAILED on error (checked below).
let mmap_ptr = unsafe {
libc::mmap(
ptr::null_mut(),
@@ -173,6 +117,8 @@ fn main() -> Result<()> {
"[3/4] mmap SUCCESS — CPU can read DMA-BUF ({:.1} MB)\n",
mmap_size as f64 / 1024.0 / 1024.0
);
// SAFETY: mmap_ptr was returned by mmap above and is not MAP_FAILED (checked);
// mmap_size matches the original mapping. POSIX munmap(2) releases the mapping.
unsafe {
libc::munmap(mmap_ptr, mmap_size);
}
@@ -205,6 +151,9 @@ fn main() -> Result<()> {
let codec_name = codec.name();
if codec_name == "libx264" {
// SAFETY: enc is a valid AVCodecContext for the not-yet-opened encoder;
// priv_data is the x264 private options struct. All CStrings live across
// both av_opt_set calls. These set the x264 "preset" and "tune" options.
unsafe {
let key = CString::new("preset").unwrap();
let val = CString::new("veryfast").unwrap();
@@ -220,6 +169,8 @@ fn main() -> Result<()> {
// Create output format context via FFI
let mut fmt_ctx_ptr: *mut ffi::AVFormatContext = ptr::null_mut();
// SAFETY: fmt_ctx_ptr is an out-parameter initialized by FFmpeg; output_cstr
// lives across the call. Returns 0 on success; we check below.
let ret = unsafe {
ffi::avformat_alloc_output_context2(
&mut fmt_ctx_ptr,
@@ -232,21 +183,30 @@ fn main() -> Result<()> {
bail!("Failed to allocate output format context: error {ret}");
}
// SAFETY: fmt_ctx_ptr is the valid output context allocated above.
// avformat_new_stream returns a pointer to a new AVStream or NULL on failure.
let stream_ptr = unsafe { ffi::avformat_new_stream(fmt_ctx_ptr, ptr::null()) };
if stream_ptr.is_null() {
bail!("Failed to create new stream");
}
// SAFETY: stream_ptr and enc_video.as_ptr() are valid pointers; codecpar is
// the output destination inside stream. avcodec_parameters_from_context copies
// encoder parameters into the stream's codecpar.
let ret =
unsafe { ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr()) };
if ret < 0 {
bail!("Failed to copy encoder parameters: error {ret}");
}
// SAFETY: stream_ptr and enc_video are valid; time_base is a plain AVRational
// field copied from encoder to stream.
unsafe {
(*stream_ptr).time_base = (*enc_video.as_ptr()).time_base;
}
// SAFETY: fmt_ctx_ptr is valid; pb is the AVIOContext slot to initialize;
// output_cstr is a valid NUL-terminated path; AVIO_FLAG_WRITE is a constant.
let ret = unsafe {
ffi::avio_open(
&mut (*fmt_ctx_ptr).pb,
@@ -261,17 +221,23 @@ fn main() -> Result<()> {
);
}
// SAFETY: fmt_ctx_ptr is fully configured (streams + pb set); NULL options
// is the default. Returns 0 on success.
let ret = unsafe { ffi::avformat_write_header(fmt_ctx_ptr, ptr::null_mut()) };
if ret < 0 {
bail!("Failed to write header: error {ret}");
}
// SAFETY: fmt_ctx_ptr is a fully initialized output context (header written).
// Output::wrap takes ownership of the pointer into a safe RAII wrapper.
let mut octx = unsafe { ff::format::context::Output::wrap(fmt_ctx_ptr) };
// Create sws_scale context: BGRZ (BGR0) -> YUV420P
let bgr0_fmt = pix_fmt(ff::format::Pixel::BGRZ);
let yuv420p_fmt = pix_fmt(ff::format::Pixel::YUV420P);
// SAFETY: all parameters are valid enum/pixel format values; NULL filters are
// allowed by FFmpeg. sws_getContext returns a heap-allocated SwsContext or NULL.
let sws_ctx = unsafe {
ffi::sws_getContext(
src_width as i32,
@@ -291,6 +257,9 @@ fn main() -> Result<()> {
}
// Allocate reusable YUV frame
// SAFETY: av_frame_alloc returns NULL only on OOM. After allocation we set
// width/height/format fields and call av_frame_get_buffer to allocate plane
// data. On failure we free the frame via av_frame_free before bailing.
let mut yuv_frame = unsafe {
let mut f = ffi::av_frame_alloc();
if f.is_null() {
@@ -349,6 +318,9 @@ fn main() -> Result<()> {
let mmap_start = Instant::now();
let frame_size = (frame.stride as usize) * (frame.height as usize);
// SAFETY: frame.fd is an open DMA-BUF owned by the frame; offset/size come
// from PipeWire's negotiated format. PROT_READ+MAP_SHARED for read-only
// DMA-BUF access. Returns MAP_FAILED on error (checked below).
let mmap_ptr = unsafe {
libc::mmap(
ptr::null_mut(),
@@ -369,8 +341,15 @@ fn main() -> Result<()> {
stats.mmap_us.push(mmap_start.elapsed().as_micros() as u64);
let scale_start = Instant::now();
// SAFETY: mmap_ptr is a valid mapping of frame_size bytes (checked above);
// constructing a read-only slice over it for the duration of sws_scale is
// sound as long as we don't hold it past munmap (we don't).
let src_data = unsafe { std::slice::from_raw_parts(mmap_ptr as *const u8, frame_size) };
// SAFETY: yuv_frame and sws_ctx are valid; src_data is a valid slice of the
// mmap'd DMA-BUF for this frame. sws_scale reads src planes (BGR0 -> YUV420P)
// and writes into yuv_frame's data planes. av_frame_make_writable ensures
// yuv_frame is not shared before writing.
unsafe {
ffi::av_frame_make_writable(yuv_frame);
@@ -391,6 +370,8 @@ fn main() -> Result<()> {
.scale_us
.push(scale_start.elapsed().as_micros() as u64);
// SAFETY: mmap_ptr was returned by mmap above and is not MAP_FAILED; frame_size
// matches the original mapping. Release before dropping frame (which closes fd).
unsafe {
libc::munmap(mmap_ptr, frame_size);
}
@@ -398,6 +379,9 @@ fn main() -> Result<()> {
let encode_start = Instant::now();
// SAFETY: yuv_frame is allocated and writable; enc_video is the opened encoder.
// Setting pts is a plain i64 field write. avcodec_send_frame submits the frame
// for encoding; returns < 0 on error (we log and continue).
unsafe {
(*yuv_frame).pts = pts;
pts += 1;
@@ -409,7 +393,7 @@ fn main() -> Result<()> {
}
}
drain_encoder(&mut enc_video, &mut octx)?;
common::drain_encoder(&mut enc_video, &mut octx)?;
stats
.encode_us
@@ -419,7 +403,7 @@ fn main() -> Result<()> {
.push(frame_start.elapsed().as_micros() as u64);
frames_encoded += 1;
if frames_encoded % 30 == 0 {
if frames_encoded.is_multiple_of(30) {
let fps = frames_encoded as f64 / total_start.elapsed().as_secs_f64();
println!(
" [{}/{}] {:.1} FPS",
@@ -431,15 +415,20 @@ fn main() -> Result<()> {
let total_elapsed = total_start.elapsed();
println!("\nFlushing encoder...");
// SAFETY: enc_video is the opened encoder; passing NULL frame signals EOF to
// drain the encoder's internal pipeline. Returns < 0 on error (ignored here).
unsafe {
ffi::avcodec_send_frame(enc_video.as_mut_ptr(), ptr::null());
}
drain_encoder(&mut enc_video, &mut octx)?;
common::drain_encoder(&mut enc_video, &mut octx)?;
octx.write_trailer()
.map_err(|e| anyhow::anyhow!("Failed to write trailer: {e}"))?;
// Cleanup
// SAFETY: yuv_frame is the allocated frame from earlier (still owned by us);
// sws_ctx is the allocated sws context. av_frame_free and sws_freeContext take
// ownership and free their respective heap allocations.
unsafe {
ffi::av_frame_free(&mut yuv_frame as *mut _);
ffi::sws_freeContext(sws_ctx);
@@ -519,32 +508,3 @@ fn main() -> Result<()> {
println!("Output written to: {}", bench_args.output);
Ok(())
}
fn drain_encoder(
enc_video: &mut ff::encoder::video::Video,
octx: &mut ff::format::context::Output,
) -> Result<()> {
loop {
let mut pkt = ff::Packet::empty();
let ret = unsafe { ffi::avcodec_receive_packet(enc_video.as_mut_ptr(), pkt.as_mut_ptr()) };
if ret < 0 {
if ret == ffi::AVERROR(ffi::EAGAIN) || ret == ffi::AVERROR_EOF {
break;
}
eprintln!("avcodec_receive_packet failed: {ret}");
break;
}
let enc_tb = enc_video.time_base();
let stream_tb = unsafe {
let streams = (*octx.as_ptr()).streams;
let st = *streams.add(0);
ff::Rational::from((*st).time_base)
};
pkt.rescale_ts(enc_tb, stream_tb);
pkt.set_stream(0);
pkt.write_interleaved(octx)
.map_err(|e| anyhow::anyhow!("write packet failed: {e}"))?;
}
Ok(())
}
+45
View File
@@ -0,0 +1,45 @@
use clap::Parser;
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
#[derive(Parser, Debug)]
#[command(
name = "sw_encode_bench",
about = "Software encoding pipeline benchmark"
)]
pub(crate) struct BenchArgs {
#[arg(short, long)]
pub(crate) output: String,
#[arg(long, default_value_t = 120)]
pub(crate) frames: u32,
#[arg(long, default_value_t = 2560)]
pub(crate) enc_width: u32,
#[arg(long, default_value_t = 1440)]
pub(crate) enc_height: u32,
}
#[derive(Default)]
pub(crate) struct FrameStats {
pub(crate) mmap_us: Vec<u64>,
pub(crate) scale_us: Vec<u64>,
pub(crate) encode_us: Vec<u64>,
pub(crate) total_us: Vec<u64>,
pub(crate) mmap_failures: u32,
}
impl FrameStats {
pub(crate) fn avg_ms(data: &[u64]) -> f64 {
if data.is_empty() {
return 0.0;
}
data.iter().sum::<u64>() as f64 / data.len() as f64 / 1000.0
}
}
pub(crate) fn pix_fmt(p: ff::format::Pixel) -> ffi::AVPixelFormat {
Into::<ffi::AVPixelFormat>::into(p)
}
File diff suppressed because it is too large Load Diff
+218
View File
@@ -0,0 +1,218 @@
// vaapi_import_bench.rs — VAAPI DMA-BUF import + GPU-side downscale benchmark
//
// Tests: Portal capture -> av_hwframe_map (ARGB sw_format) -> transfer -> sw encode
//
// Usage: cargo run --bin vaapi_import_bench -- --output /tmp/vaapi_bench.mp4
use std::os::fd::AsRawFd;
use std::path::Path;
use std::ptr;
use std::time::Instant;
use anyhow::Result;
use clap::Parser;
use ffmpeg_next as ff;
use wl_webrtc::args::Args;
use wl_webrtc::avhw::{import_dma_buf_to_vaapi, AvHwDevCtx, AvHwFrameCtx};
use wl_webrtc::cap_portal::CapPortal;
#[path = "../common/mod.rs"]
mod common;
mod pipeline_cpu;
mod pipeline_gpu;
mod software;
mod stats;
mod util;
use pipeline_cpu::run_cpu_pipeline;
use pipeline_gpu::run_gpu_pipeline;
use stats::{BenchArgs, PipelineMode};
use util::{output_for_mode, print_comparison, print_detailed_results};
fn main() -> Result<()> {
let bench_args = BenchArgs::parse();
println!("=== VAAPI Import Benchmark ===");
println!("Output: {}", bench_args.output);
println!("Target frames: {}", bench_args.frames);
println!(
"Encode resolution: {}x{}",
bench_args.enc_width, bench_args.enc_height
);
println!("DRM device: {}", bench_args.drm_device);
println!();
ff::init()?;
println!("[1/3] Requesting screen capture via XDG Portal...");
println!(" (Select a screen to share in the portal dialog)");
let portal_args = Args {
output: Some(bench_args.output.clone()),
output_name: None,
fps: 60,
codec: "h264".to_string(),
hw_accel: "vaapi".to_string(),
drm_device: None,
bitrate: None,
max_bitrate: 8_000_000,
gop_size: None,
verbose: false,
backend: Some("portal".to_string()),
port: 0,
no_persist: false,
stats: false,
};
let cap = CapPortal::new(&portal_args)?;
println!("[1/3] Portal connected, PipeWire stream active\n");
println!("[2/3] Waiting for first frame from PipeWire...");
let first_frame = common::receive_first_frame(&cap)?;
let src_width = first_frame.width;
let src_height = first_frame.height;
let src_format = first_frame.format;
println!(
"[2/3] First frame: {}x{}, format=0x{:08X}, stride={}, modifier=0x{:X}",
src_width, src_height, src_format, first_frame.stride, first_frame.modifier
);
println!("\n[2/3] Testing av_hwframe_map with sw_format=BGRA...");
println!(
" DRM format chain: PipeWire BGRA -> DRM_FORMAT_ARGB8888 (0x{:08X}) -> VA_FOURCC_BGRA -> AV_PIX_FMT_BGRA",
src_format
);
let drm_device = Path::new(&bench_args.drm_device);
let hw_dev = AvHwDevCtx::new_vaapi(drm_device)?;
println!(" VAAPI device context created OK");
let frames_ctx =
AvHwFrameCtx::for_capture(&hw_dev, src_width, src_height, ff::format::Pixel::BGRA)?;
println!(" VAAPI frames context created OK (sw_format=BGRA)");
// SAFETY: delegates to avhw::import_dma_buf_to_vaapi (itself an unsafe fn).
// frames_ctx is a valid AVBufferRef from AvHwFrameCtx::for_capture above;
// `first_frame` is the PipeWire-formatted PwDmaBufFrame whose metadata the
// function reads directly. See that function's own SAFETY contract for the
// full rationale.
let vaapi_frame = unsafe { import_dma_buf_to_vaapi(frames_ctx.as_ptr(), &first_frame) };
match &vaapi_frame {
Ok(_) => {
println!(" Result: SUCCESS — av_hwframe_map imported DMA-BUF to VAAPI surface!");
}
Err(e) => {
println!(" Result: FAILED");
println!(" Error: {e}");
println!();
println!(" Possible causes:");
println!(" - sw_format mismatch (current: BGRA)");
println!(" - DRM format modifier not supported by VAAPI");
println!(" - VAAPI driver doesn't support DMA-BUF import for this format");
println!();
println!(" Falling back to mmap readback test for comparison...");
let mmap_size = (first_frame.stride as usize) * (first_frame.height as usize);
let mmap_start = Instant::now();
// SAFETY: first_frame.fd is an open DMA-BUF; offset/size from PipeWire.
// PROT_READ+MAP_SHARED is the standard read-only DMA-BUF mapping. Returns
// MAP_FAILED on error (checked below).
let mmap_ptr = unsafe {
libc::mmap(
ptr::null_mut(),
mmap_size,
libc::PROT_READ,
libc::MAP_SHARED,
first_frame.fd.as_raw_fd(),
first_frame.offset as i64,
)
};
let mmap_elapsed = mmap_start.elapsed();
if mmap_ptr == libc::MAP_FAILED {
let errno = std::io::Error::last_os_error();
println!(" mmap also FAILED: {errno}");
} else {
println!(
" mmap SUCCESS: {:.1} MB, setup in {:.2}ms",
mmap_size as f64 / 1024.0 / 1024.0,
mmap_elapsed.as_secs_f64() * 1000.0
);
// SAFETY: mmap_ptr is a valid mapping (MAP_FAILED path was handled
// above); mmap_size matches the original mapping. POSIX munmap(2).
unsafe {
libc::munmap(mmap_ptr, mmap_size);
}
}
println!();
println!("=== Benchmark ended: av_hwframe_map import FAILED ===");
println!("Fix the import issue before proceeding to GPU downscale tests.");
return Ok(());
}
}
drop(vaapi_frame);
drop(first_frame);
println!("\n[3/3] Benchmarking selected pipeline(s)...");
let enc_width = bench_args.enc_width;
let enc_height = bench_args.enc_height;
let split_outputs = bench_args.mode == PipelineMode::Both;
let mut cpu_stats = None;
let mut gpu_stats = None;
if matches!(bench_args.mode, PipelineMode::Cpu | PipelineMode::Both) {
let output = output_for_mode(&bench_args.output, PipelineMode::Cpu, split_outputs);
cpu_stats = Some(run_cpu_pipeline(
&cap,
&frames_ctx,
&output,
bench_args.frames,
src_width,
src_height,
enc_width,
enc_height,
)?);
}
if matches!(bench_args.mode, PipelineMode::Gpu | PipelineMode::Both) {
let output = output_for_mode(&bench_args.output, PipelineMode::Gpu, split_outputs);
gpu_stats = Some(run_gpu_pipeline(
&cap,
&hw_dev,
&frames_ctx,
&output,
bench_args.frames,
src_width,
src_height,
enc_width,
enc_height,
)?);
}
if let Some(stats) = cpu_stats.as_ref() {
print_detailed_results("CPU", stats, src_width, src_height, enc_width, enc_height);
}
if let Some(stats) = gpu_stats.as_ref() {
print_detailed_results("GPU", stats, src_width, src_height, enc_width, enc_height);
}
print_comparison(cpu_stats.as_ref(), gpu_stats.as_ref());
if cpu_stats
.as_ref()
.into_iter()
.chain(gpu_stats.as_ref())
.any(|stats| stats.achieved_fps() < 30.0 && stats.frames_encoded > 0)
{
println!("NOTE: At least one achieved FPS result is below 30 FPS target.");
}
Ok(())
}
+152
View File
@@ -0,0 +1,152 @@
use std::path::Path;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next::ffi;
use wl_webrtc::avhw::{av_err_to_string, AvHwFrameCtx};
use wl_webrtc::cap_portal::{CapPortal, PwCtrlEvent};
use crate::pipeline_gpu::import_frame;
use crate::software::{
create_software_encoder, create_sws_context, encode_yuv_frame, finish_encoder,
};
use crate::stats::FrameStats;
#[allow(clippy::too_many_arguments)]
pub(crate) fn run_cpu_pipeline(
cap: &CapPortal,
frames_ctx: &AvHwFrameCtx,
output: &str,
frames: u32,
src_width: u32,
src_height: u32,
enc_width: u32,
enc_height: u32,
) -> Result<FrameStats> {
let mut encoder = create_software_encoder(Path::new(output), enc_width, enc_height)?;
let sws_ctx = create_sws_context(
src_width,
src_height,
ffi::AVPixelFormat::AV_PIX_FMT_BGRA,
enc_width,
enc_height,
)?;
println!(
" Encoder: {}, {}x{} YUV420P",
encoder.codec_name, enc_width, enc_height
);
println!(" Output: {output}");
println!(" CPU Pipeline: DMA-BUF 4K BGRA -> av_hwframe_map -> av_hwframe_transfer_data -> sws_scale -> YUV420P 2K -> encode\n");
let mut stats = FrameStats {
codec_name: encoder.codec_name.clone(),
output_path: output.to_string(),
..FrameStats::default()
};
let total_start = Instant::now();
let mut pts: i64 = 0;
while stats.frames_encoded < frames {
if let Ok(ctrl) = cap.event_receiver().try_recv() {
match ctrl {
PwCtrlEvent::StreamEnded => break,
PwCtrlEvent::Error(e) => bail!(
"PipeWire error after {} CPU frames: {e}",
stats.frames_encoded
),
PwCtrlEvent::FormatChanged { .. } => {}
}
}
let frame = match cap
.frame_receiver()
.recv_timeout(std::time::Duration::from_secs(5))
{
Ok(f) => f,
Err(_) => break,
};
let frame_start = Instant::now();
let t_import = Instant::now();
let vaapi_frame = match import_frame(frames_ctx, &frame) {
Ok(f) => f,
Err(e) => {
stats.import_failures += 1;
if stats.import_failures <= 3 {
eprintln!("CPU frame {}: import failed: {e}", stats.frames_encoded);
}
continue;
}
};
let import_us = t_import.elapsed().as_micros() as u64;
let t_transfer = Instant::now();
// SAFETY: sw_frame is allocated by FFmpeg and freed on all paths below.
let mut sw_frame = unsafe { ffi::av_frame_alloc() };
if sw_frame.is_null() {
bail!("CPU frame {}: av_frame_alloc failed", stats.frames_encoded);
}
// SAFETY: sw_frame is an allocated destination; vaapi_frame is a valid VAAPI source frame.
let transfer_ret =
unsafe { ffi::av_hwframe_transfer_data(sw_frame, vaapi_frame.as_ptr(), 0) };
if transfer_ret < 0 {
// SAFETY: sw_frame was allocated above and has not been freed yet.
unsafe { ffi::av_frame_free(&mut sw_frame) };
bail!(
"CPU frame {}: av_hwframe_transfer_data failed: {} ({})",
stats.frames_encoded,
transfer_ret,
av_err_to_string(transfer_ret)
);
}
let transfer_us = t_transfer.elapsed().as_micros() as u64;
let t_scale = Instant::now();
// SAFETY: sw_frame contains transferred BGRA data; encoder.yuv_frame is writable YUV420P
// at the configured output dimensions; sws_ctx converts and downscales between them.
unsafe {
ffi::av_frame_make_writable(encoder.yuv_frame);
ffi::sws_scale(
sws_ctx.0,
(*sw_frame).data.as_ptr() as *const *const u8,
(*sw_frame).linesize.as_ptr(),
0,
(*sw_frame).height,
(*encoder.yuv_frame).data.as_ptr() as *mut *mut u8,
(*encoder.yuv_frame).linesize.as_ptr(),
);
}
let scale_us = t_scale.elapsed().as_micros() as u64;
// SAFETY: sw_frame was allocated above and is no longer needed after scaling.
unsafe { ffi::av_frame_free(&mut sw_frame) };
let encode_us = encode_yuv_frame(&mut encoder, &mut pts)?;
let total_us = frame_start.elapsed().as_micros() as u64;
stats.import_us.push(import_us);
stats.transfer_us.push(transfer_us);
stats.scale_us.push(scale_us);
stats.encode_us.push(encode_us);
stats.total_us.push(total_us);
stats.frames_encoded += 1;
if stats.frames_encoded <= 3 || stats.frames_encoded.is_multiple_of(30) {
println!(
" CPU frame {:>4}/{frames}: import={:.2}ms transfer={:.2}ms scale={:.2}ms encode={:.2}ms total={:.2}ms",
stats.frames_encoded,
import_us as f64 / 1000.0,
transfer_us as f64 / 1000.0,
scale_us as f64 / 1000.0,
encode_us as f64 / 1000.0,
total_us as f64 / 1000.0,
);
}
}
finish_encoder(encoder)?;
stats.elapsed_secs = total_start.elapsed().as_secs_f64();
Ok(stats)
}
+250
View File
@@ -0,0 +1,250 @@
use std::path::Path;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use wl_webrtc::avhw::{av_err_to_string, import_dma_buf_to_vaapi, AvHwDevCtx, AvHwFrameCtx};
use wl_webrtc::cap_portal::{CapPortal, PwCtrlEvent};
use crate::software::{
create_software_encoder, create_sws_context, encode_yuv_frame, finish_encoder,
};
use crate::stats::FrameStats;
pub(crate) fn import_frame(
frames_ctx: &AvHwFrameCtx,
frame: &wl_webrtc::cap_portal::PwDmaBufFrame,
) -> Result<ff::frame::Video> {
// SAFETY: frames_ctx is a live VAAPI frames context configured for the capture format; frame
// carries a valid DMA-BUF fd and metadata from PipeWire for the duration of the call.
// SAFETY: frames_ctx is a valid VAAPI frames context; `frame` carries the
// DMA-BUF metadata read by the function.
unsafe { import_dma_buf_to_vaapi(frames_ctx.as_ptr(), frame) }
}
fn build_gpu_filter_graph(
hw_dev: &AvHwDevCtx,
frames_rgb: &AvHwFrameCtx,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
) -> Result<ff::filter::Graph> {
let mut graph = ff::filter::Graph::new();
let buffersrc =
ff::filter::find("buffer").ok_or_else(|| anyhow::anyhow!("filter 'buffer' not found"))?;
let buffersink = ff::filter::find("buffersink")
.ok_or_else(|| anyhow::anyhow!("filter 'buffersink' not found"))?;
let scale_vaapi = ff::filter::find("scale_vaapi")
.ok_or_else(|| anyhow::anyhow!("filter 'scale_vaapi' not found"))?;
// pix_fmt must be set via av_buffersrc_parameters_set (below), not in args —
// FFmpeg 8.0+ rejects HW pixel formats during init() if hw_frames_ctx is missing.
// Use a placeholder SW format here; it gets overridden by parameters_set below.
let args = format!(
"video_size={}x{}:pix_fmt=bgra:time_base=1/60:pixel_aspect=1/1",
width, height,
);
let mut src_ctx = graph.add(&buffersrc, "in", &args)?;
// SAFETY: Allocate buffersrc parameters, attach a ref-counted hw_frames_ctx compatible with
// imported VAAPI BGRA frames, apply it, then free only the parameter struct (not the ref).
let par = unsafe { ffi::av_buffersrc_parameters_alloc() };
if par.is_null() {
bail!("av_buffersrc_parameters_alloc returned null");
}
// SAFETY: par and src_ctx are valid; frames_rgb.ref_clone returns an owned AVBufferRef.
unsafe {
(*par).format = Into::<ffi::AVPixelFormat>::into(ff::format::Pixel::VAAPI) as i32;
(*par).width = width as i32;
(*par).height = height as i32;
(*par).time_base = ffi::AVRational { num: 1, den: 60 };
(*par).hw_frames_ctx = frames_rgb.ref_clone();
let ret = ffi::av_buffersrc_parameters_set(src_ctx.as_mut_ptr(), par);
ffi::av_free(par as *mut _);
if ret < 0 {
bail!("av_buffersrc_parameters_set failed: error {ret}");
}
}
let mut scale_ctx = graph.add(
&scale_vaapi,
"scale",
&format!("{enc_width}:{enc_height}:format=nv12"),
)?;
// SAFETY: scale_vaapi uses this ref-counted VAAPI device context while graph is alive.
unsafe {
(*scale_ctx.as_mut_ptr()).hw_device_ctx = hw_dev.ref_clone();
}
let mut sink_ctx = graph.add(&buffersink, "out", "")?;
src_ctx.link(0, &mut scale_ctx, 0);
scale_ctx.link(0, &mut sink_ctx, 0);
graph
.validate()
.map_err(|e| anyhow::anyhow!("GPU filter graph validation failed: {e}"))?;
Ok(graph)
}
#[allow(clippy::too_many_arguments)]
pub(crate) fn run_gpu_pipeline(
cap: &CapPortal,
hw_dev: &AvHwDevCtx,
frames_ctx: &AvHwFrameCtx,
output: &str,
frames: u32,
src_width: u32,
src_height: u32,
enc_width: u32,
enc_height: u32,
) -> Result<FrameStats> {
let mut encoder = create_software_encoder(Path::new(output), enc_width, enc_height)?;
let format_ctx = create_sws_context(
enc_width,
enc_height,
ffi::AVPixelFormat::AV_PIX_FMT_NV12,
enc_width,
enc_height,
)?;
let mut graph = build_gpu_filter_graph(
hw_dev, frames_ctx, src_width, src_height, enc_width, enc_height,
)?;
println!(
" Encoder: {}, {}x{} YUV420P",
encoder.codec_name, enc_width, enc_height
);
println!(" Output: {output}");
println!(" GPU Pipeline: DMA-BUF 4K BGRA -> av_hwframe_map -> scale_vaapi 2K NV12 -> transfer small NV12 -> sws_scale format-only -> encode\n");
let mut stats = FrameStats {
codec_name: encoder.codec_name.clone(),
output_path: output.to_string(),
..FrameStats::default()
};
let total_start = Instant::now();
let mut pts: i64 = 0;
while stats.frames_encoded < frames {
if let Ok(ctrl) = cap.event_receiver().try_recv() {
match ctrl {
PwCtrlEvent::StreamEnded => break,
PwCtrlEvent::Error(e) => bail!(
"PipeWire error after {} GPU frames: {e}",
stats.frames_encoded
),
PwCtrlEvent::FormatChanged { .. } => {}
}
}
let frame = match cap
.frame_receiver()
.recv_timeout(std::time::Duration::from_secs(5))
{
Ok(f) => f,
Err(_) => break,
};
let frame_start = Instant::now();
let t_import = Instant::now();
let vaapi_frame = match import_frame(frames_ctx, &frame) {
Ok(f) => f,
Err(e) => {
stats.import_failures += 1;
if stats.import_failures <= 3 {
eprintln!("GPU frame {}: import failed: {e}", stats.frames_encoded);
}
continue;
}
};
let import_us = t_import.elapsed().as_micros() as u64;
let t_filter = Instant::now();
let mut filter_src_ctx = graph.get("in").unwrap();
let mut filter_src = filter_src_ctx.source();
let mut filter_sink_ctx = graph.get("out").unwrap();
let mut filter_sink = filter_sink_ctx.sink();
filter_src
.add(&vaapi_frame)
.map_err(|e| anyhow::anyhow!("GPU filter source add failed: {e}"))?;
let mut filtered = ff::frame::Video::empty();
match filter_sink.frame(&mut filtered) {
Ok(()) => {}
Err(ff::Error::Other { errno }) if errno == ffi::EAGAIN => continue,
Err(e) => bail!("GPU filter sink get frame failed: {e}"),
}
let filter_us = t_filter.elapsed().as_micros() as u64;
let t_transfer = Instant::now();
// SAFETY: sw_nv12 is allocated by FFmpeg and freed after format conversion.
let mut sw_nv12 = unsafe { ffi::av_frame_alloc() };
if sw_nv12.is_null() {
bail!("GPU frame {}: av_frame_alloc failed", stats.frames_encoded);
}
// SAFETY: sw_nv12 is an allocated destination; filtered is a valid 2K NV12 VAAPI frame.
let transfer_ret = unsafe { ffi::av_hwframe_transfer_data(sw_nv12, filtered.as_ptr(), 0) };
if transfer_ret < 0 {
// SAFETY: sw_nv12 was allocated above and has not been freed yet.
unsafe { ffi::av_frame_free(&mut sw_nv12) };
bail!(
"GPU frame {}: av_hwframe_transfer_data failed: {} ({})",
stats.frames_encoded,
transfer_ret,
av_err_to_string(transfer_ret)
);
}
let transfer_us = t_transfer.elapsed().as_micros() as u64;
let t_format = Instant::now();
// SAFETY: sw_nv12 contains CPU-side NV12 at enc dimensions; encoder.yuv_frame is writable
// YUV420P at the same dimensions, so sws_scale performs only chroma deinterleave/format conversion.
unsafe {
ffi::av_frame_make_writable(encoder.yuv_frame);
ffi::sws_scale(
format_ctx.0,
(*sw_nv12).data.as_ptr() as *const *const u8,
(*sw_nv12).linesize.as_ptr(),
0,
(*sw_nv12).height,
(*encoder.yuv_frame).data.as_ptr() as *mut *mut u8,
(*encoder.yuv_frame).linesize.as_ptr(),
);
}
let format_us = t_format.elapsed().as_micros() as u64;
// SAFETY: sw_nv12 was allocated above and is no longer needed.
unsafe { ffi::av_frame_free(&mut sw_nv12) };
let encode_us = encode_yuv_frame(&mut encoder, &mut pts)?;
let total_us = frame_start.elapsed().as_micros() as u64;
stats.import_us.push(import_us);
stats.filter_us.push(filter_us);
stats.transfer_us.push(transfer_us);
stats.format_us.push(format_us);
stats.encode_us.push(encode_us);
stats.total_us.push(total_us);
stats.frames_encoded += 1;
if stats.frames_encoded <= 3 || stats.frames_encoded.is_multiple_of(30) {
println!(
" GPU frame {:>4}/{frames}: import={:.2}ms filter={:.2}ms transfer={:.2}ms format={:.2}ms encode={:.2}ms total={:.2}ms",
stats.frames_encoded,
import_us as f64 / 1000.0,
filter_us as f64 / 1000.0,
transfer_us as f64 / 1000.0,
format_us as f64 / 1000.0,
encode_us as f64 / 1000.0,
total_us as f64 / 1000.0,
);
}
}
finish_encoder(encoder)?;
stats.elapsed_secs = total_start.elapsed().as_secs_f64();
Ok(stats)
}
+228
View File
@@ -0,0 +1,228 @@
use std::ffi::CString;
use std::path::Path;
use std::ptr;
use std::time::Instant;
use anyhow::{bail, Result};
use ffmpeg_next as ff;
use ffmpeg_next::ffi;
use crate::common::drain_encoder;
pub(crate) struct SoftwareEncoder {
pub(crate) enc_video: ff::codec::encoder::video::Video,
pub(crate) octx: ff::format::context::Output,
pub(crate) yuv_frame: *mut ffi::AVFrame,
pub(crate) codec_name: String,
}
impl Drop for SoftwareEncoder {
fn drop(&mut self) {
// SAFETY: yuv_frame is allocated by av_frame_alloc in create_software_encoder and
// owned exclusively by this SoftwareEncoder.
unsafe {
ffi::av_frame_free(&mut self.yuv_frame);
}
}
}
pub(crate) struct SwsContext(pub(crate) *mut ffi::SwsContext);
impl Drop for SwsContext {
fn drop(&mut self) {
// SAFETY: Context is either null or returned by sws_getContext and owned here.
unsafe {
ffi::sws_freeContext(self.0);
}
}
}
pub(crate) fn create_software_encoder(
output_path: &Path,
width: u32,
height: u32,
) -> Result<SoftwareEncoder> {
let output_cstr = CString::new(output_path.to_str().unwrap())?;
let codec = ff::encoder::find_by_name("libx264")
.or_else(|| ff::encoder::find_by_name("libopenh264"))
.ok_or_else(|| {
anyhow::anyhow!("No H.264 software encoder found (tried libx264, libopenh264)")
})?;
let codec_name = codec.name().to_string();
let mut enc = {
let ctx = ff::codec::Context::new_with_codec(codec);
ctx.encoder().video()?
};
enc.set_width(width);
enc.set_height(height);
enc.set_format(ff::format::Pixel::YUV420P);
enc.set_time_base(ff::Rational::new(1, 60));
enc.set_max_b_frames(0);
enc.set_gop(60);
if codec_name == "libx264" {
// SAFETY: priv_data belongs to the not-yet-opened encoder context. Option strings are
// valid NUL-terminated C strings for the duration of each av_opt_set call.
unsafe {
let key = CString::new("preset").unwrap();
let val = CString::new("veryfast").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
let key = CString::new("tune").unwrap();
let val = CString::new("zerolatency").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
}
}
let opened = enc.open()?;
let enc_video = opened.0;
let use_null_muxer = output_path
.to_str()
.map(|s| s.contains("null"))
.unwrap_or(false);
let fmt_name = if use_null_muxer {
CString::new("null").unwrap()
} else {
CString::new("").unwrap()
};
let fmt_name_ptr = if use_null_muxer {
fmt_name.as_ptr()
} else {
ptr::null()
};
let mut fmt_ctx_ptr: *mut ffi::AVFormatContext = ptr::null_mut();
// SAFETY: fmt_ctx_ptr is an out parameter initialized by FFmpeg; output_cstr and fmt_name live
// across the call.
let ret = unsafe {
ffi::avformat_alloc_output_context2(
&mut fmt_ctx_ptr,
ptr::null_mut(),
fmt_name_ptr,
output_cstr.as_ptr(),
)
};
if ret < 0 || fmt_ctx_ptr.is_null() {
bail!("Failed to allocate output format context: error {ret}");
}
// SAFETY: fmt_ctx_ptr is a valid output context allocated above.
let stream_ptr = unsafe { ffi::avformat_new_stream(fmt_ctx_ptr, ptr::null()) };
if stream_ptr.is_null() {
bail!("Failed to create output stream");
}
// SAFETY: stream and codec context pointers are valid; parameters are copied into stream.
let ret =
unsafe { ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr()) };
if ret < 0 {
bail!("Failed to copy codec parameters: error {ret}");
}
// SAFETY: fmt_ctx_ptr is valid; pb is initialized for non-NOFILE muxers.
unsafe {
if (*(*fmt_ctx_ptr).oformat).flags & ffi::AVFMT_NOFILE == 0 {
let ret = ffi::avio_open(
&mut (*fmt_ctx_ptr).pb,
output_cstr.as_ptr(),
ffi::AVIO_FLAG_WRITE,
);
if ret < 0 {
bail!("Failed to open output file: error {ret}");
}
}
}
// SAFETY: fmt_ctx_ptr is a fully configured output context.
let ret = unsafe { ffi::avformat_write_header(fmt_ctx_ptr, ptr::null_mut()) };
if ret < 0 {
bail!("Failed to write header: error {ret}");
}
// SAFETY: ownership of fmt_ctx_ptr transfers into ffmpeg-next Output wrapper.
let octx = unsafe { ff::format::context::Output::wrap(fmt_ctx_ptr) };
// SAFETY: Allocate and configure an owned writable YUV420P frame for encoder input.
let yuv_frame = unsafe {
let mut f = ffi::av_frame_alloc();
if f.is_null() {
bail!("av_frame_alloc failed");
}
(*f).width = width as i32;
(*f).height = height as i32;
(*f).format = ffi::AVPixelFormat::AV_PIX_FMT_YUV420P as i32;
let r = ffi::av_frame_get_buffer(f, 0);
if r < 0 {
ffi::av_frame_free(&mut f);
bail!("av_frame_get_buffer failed: {r}");
}
f
};
Ok(SoftwareEncoder {
enc_video,
octx,
yuv_frame,
codec_name,
})
}
pub(crate) fn create_sws_context(
src_width: u32,
src_height: u32,
src_fmt: ffi::AVPixelFormat,
dst_width: u32,
dst_height: u32,
) -> Result<SwsContext> {
// SAFETY: sws_getContext creates an owned scaler context for the provided dimensions/formats.
let ctx = unsafe {
ffi::sws_getContext(
src_width as i32,
src_height as i32,
src_fmt,
dst_width as i32,
dst_height as i32,
ffi::AVPixelFormat::AV_PIX_FMT_YUV420P,
2,
ptr::null_mut(),
ptr::null_mut(),
ptr::null_mut(),
)
};
if ctx.is_null() {
bail!("Failed to create sws_scale context");
}
Ok(SwsContext(ctx))
}
pub(crate) fn encode_yuv_frame(encoder: &mut SoftwareEncoder, pts: &mut i64) -> Result<u64> {
let t_encode = Instant::now();
// SAFETY: yuv_frame is allocated, writable, and formatted as the encoder's configured
// YUV420P input frame. FFmpeg consumes but does not take ownership.
unsafe {
(*encoder.yuv_frame).pts = *pts;
*pts += 1;
let r = ffi::avcodec_send_frame(encoder.enc_video.as_mut_ptr(), encoder.yuv_frame);
if r < 0 {
bail!("avcodec_send_frame failed: {r}");
}
}
drain_encoder(&mut encoder.enc_video, &mut encoder.octx)?;
Ok(t_encode.elapsed().as_micros() as u64)
}
pub(crate) fn finish_encoder(mut encoder: SoftwareEncoder) -> Result<()> {
// SAFETY: Sending a null frame flushes the encoder; context remains owned by encoder.
unsafe {
ffi::avcodec_send_frame(encoder.enc_video.as_mut_ptr(), ptr::null());
}
drain_encoder(&mut encoder.enc_video, &mut encoder.octx)?;
encoder
.octx
.write_trailer()
.map_err(|e| anyhow::anyhow!("Failed to write trailer: {e}"))?;
Ok(())
}
+76
View File
@@ -0,0 +1,76 @@
use clap::{Parser, ValueEnum};
#[derive(Parser, Debug)]
#[command(name = "vaapi_import_bench", about = "VAAPI DMA-BUF import benchmark")]
pub(crate) struct BenchArgs {
#[arg(short, long)]
pub(crate) output: String,
#[arg(long, default_value_t = 60)]
pub(crate) frames: u32,
#[arg(long, default_value_t = 2560)]
pub(crate) enc_width: u32,
#[arg(long, default_value_t = 1440)]
pub(crate) enc_height: u32,
#[arg(long, default_value = "/dev/dri/renderD128")]
pub(crate) drm_device: String,
#[arg(long, value_enum, default_value_t = PipelineMode::Both)]
pub(crate) mode: PipelineMode,
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, ValueEnum)]
pub(crate) enum PipelineMode {
Cpu,
Gpu,
Both,
}
#[derive(Default)]
pub(crate) struct FrameStats {
pub(crate) import_us: Vec<u64>,
pub(crate) filter_us: Vec<u64>,
pub(crate) transfer_us: Vec<u64>,
pub(crate) scale_us: Vec<u64>,
pub(crate) format_us: Vec<u64>,
pub(crate) encode_us: Vec<u64>,
pub(crate) total_us: Vec<u64>,
pub(crate) import_failures: u32,
pub(crate) frames_encoded: u32,
pub(crate) elapsed_secs: f64,
pub(crate) codec_name: String,
pub(crate) output_path: String,
}
impl FrameStats {
pub(crate) fn avg_ms(data: &[u64]) -> f64 {
if data.is_empty() {
return 0.0;
}
data.iter().sum::<u64>() as f64 / data.len() as f64 / 1000.0
}
pub(crate) fn avg_total_ms(&self) -> f64 {
Self::avg_ms(&self.total_us)
}
pub(crate) fn achieved_fps(&self) -> f64 {
if self.frames_encoded > 0 && self.elapsed_secs > 0.0 {
self.frames_encoded as f64 / self.elapsed_secs
} else {
0.0
}
}
pub(crate) fn theoretical_fps(&self) -> f64 {
let avg = self.avg_total_ms();
if avg > 0.0 {
1000.0 / avg
} else {
0.0
}
}
}
+107
View File
@@ -0,0 +1,107 @@
use std::path::Path;
use crate::stats::{FrameStats, PipelineMode};
pub(crate) fn output_for_mode(base: &str, mode: PipelineMode, split: bool) -> String {
if !split || base.contains("null") {
return base.to_string();
}
let path = Path::new(base);
let suffix = match mode {
PipelineMode::Cpu => "cpu",
PipelineMode::Gpu => "gpu",
PipelineMode::Both => unreachable!(),
};
let file_name = path.file_name().and_then(|s| s.to_str()).unwrap_or(base);
let split_name = if let Some((stem, ext)) = file_name.rsplit_once('.') {
format!("{stem}.{suffix}.{ext}")
} else {
format!("{file_name}.{suffix}")
};
path.with_file_name(split_name)
.to_string_lossy()
.into_owned()
}
pub(crate) fn print_detailed_results(
label: &str,
stats: &FrameStats,
src_width: u32,
src_height: u32,
enc_width: u32,
enc_height: u32,
) {
println!();
println!("=== {label} Pipeline Results ===");
println!("Capture resolution: {}x{}", src_width, src_height);
println!("Encode resolution: {}x{}", enc_width, enc_height);
println!("Frames encoded: {}", stats.frames_encoded);
println!("Total time: {:.2}s", stats.elapsed_secs);
println!("Output: {}", stats.output_path);
if stats.import_failures > 0 {
println!("Import failures: {}", stats.import_failures);
}
println!(
"import avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.import_us)
);
if !stats.filter_us.is_empty() {
println!(
"filter avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.filter_us)
);
}
println!(
"transfer avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.transfer_us)
);
if !stats.scale_us.is_empty() {
println!(
"scale avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.scale_us)
);
}
if !stats.format_us.is_empty() {
println!(
"format avg: {:.2} ms/frame",
FrameStats::avg_ms(&stats.format_us)
);
}
println!(
"encode ({}): {:.2} ms/frame",
stats.codec_name,
FrameStats::avg_ms(&stats.encode_us)
);
println!("total avg: {:.2} ms/frame", stats.avg_total_ms());
println!("achieved FPS: {:.1}", stats.achieved_fps());
println!("max theoretical: {:.1} FPS", stats.theoretical_fps());
}
pub(crate) fn print_comparison(cpu: Option<&FrameStats>, gpu: Option<&FrameStats>) {
println!();
println!("=== Pipeline Comparison ===");
if let Some(s) = cpu {
println!(
"CPU: import={:.2}ms transfer={:.2}ms scale={:.2}ms encode={:.2}ms total={:.2}ms ({:.1} FPS)",
FrameStats::avg_ms(&s.import_us),
FrameStats::avg_ms(&s.transfer_us),
FrameStats::avg_ms(&s.scale_us),
FrameStats::avg_ms(&s.encode_us),
s.avg_total_ms(),
s.theoretical_fps(),
);
}
if let Some(s) = gpu {
println!(
"GPU: import={:.2}ms filter={:.2}ms transfer={:.2}ms format={:.2}ms encode={:.2}ms total={:.2}ms ({:.1} FPS)",
FrameStats::avg_ms(&s.import_us),
FrameStats::avg_ms(&s.filter_us),
FrameStats::avg_ms(&s.transfer_us),
FrameStats::avg_ms(&s.format_us),
FrameStats::avg_ms(&s.encode_us),
s.avg_total_ms(),
s.theoretical_fps(),
);
}
}
+39 -1086
View File
File diff suppressed because it is too large Load Diff
+73
View File
@@ -0,0 +1,73 @@
/// 将 PipeWire SPA 视频格式转换为 DRM FourCC 格式
///
/// PipeWire 使用自己的 VideoFormat 枚举,而 DRM/KMS 使用 FourCC 格式标识。
/// 此函数建立了两者之间的映射关系。
///
/// 支持的格式:
/// 不支持的格式返回 0
/// DRM 格式名描述像素值位布局(大端序),而非内存字节序。
/// 例如 DRM_FORMAT_ARGB8888 在小端 x86 上内存为 [B,G,R,A] = PipeWire BGRA。
pub(super) fn spa_to_drm_fourcc(format: libspa::param::video::VideoFormat) -> u32 {
use drm_fourcc::DrmFourcc;
use libspa::param::video::VideoFormat;
match format {
VideoFormat::BGRA => DrmFourcc::Argb8888 as u32,
VideoFormat::BGRx => DrmFourcc::Xrgb8888 as u32,
VideoFormat::RGBA => DrmFourcc::Abgr8888 as u32,
VideoFormat::RGBx => DrmFourcc::Xbgr8888 as u32,
VideoFormat::ARGB => DrmFourcc::Bgra8888 as u32,
VideoFormat::xRGB => DrmFourcc::Bgrx8888 as u32,
VideoFormat::ABGR => DrmFourcc::Rgba8888 as u32,
VideoFormat::xBGR => DrmFourcc::Rgbx8888 as u32,
_ => 0,
}
}
#[cfg(test)]
mod tests {
use super::*;
use drm_fourcc::DrmFourcc;
#[test]
fn spa_to_drm_fourcc_all_32bit() {
use libspa::param::video::VideoFormat;
assert_eq!(
spa_to_drm_fourcc(VideoFormat::BGRA),
DrmFourcc::Argb8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::BGRx),
DrmFourcc::Xrgb8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::RGBA),
DrmFourcc::Abgr8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::RGBx),
DrmFourcc::Xbgr8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::ARGB),
DrmFourcc::Bgra8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::xRGB),
DrmFourcc::Bgrx8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::ABGR),
DrmFourcc::Rgba8888 as u32
);
assert_eq!(
spa_to_drm_fourcc(VideoFormat::xBGR),
DrmFourcc::Rgbx8888 as u32
);
}
#[test]
fn spa_to_drm_fourcc_unsupported() {
use libspa::param::video::VideoFormat;
assert_eq!(spa_to_drm_fourcc(VideoFormat::NV12), 0);
}
}
+18
View File
@@ -0,0 +1,18 @@
/// Log an actionable diagnostic when a Portal phase times out.
///
/// Mirrors the message format from `backend_detect.rs::log_portal_unresponsive`
/// but additionally suggests `--no-persist` when the timeout occurred in a
/// phase that was using a restore token.
pub(super) fn log_portal_phase_timeout(phase: &str, used_restore_token: bool) {
let persist_hint = if used_restore_token {
" If this recurs, try: wl-webrtc --no-persist"
} else {
""
};
tracing::error!(
"Portal service did not respond within timeout while {phase}. \
This usually means xdg-desktop-portal or xdg-desktop-portal-kde is stuck. \
Try: systemctl --user restart xdg-desktop-portal xdg-desktop-portal-kde, \
then re-run wl-webrtc.{persist_hint}"
);
}
+446
View File
@@ -0,0 +1,446 @@
use std::os::fd::{AsRawFd, FromRawFd, OwnedFd};
use std::sync::atomic::AtomicU64;
use std::sync::atomic::Ordering;
use std::sync::Arc;
use std::thread::{self, JoinHandle};
use anyhow::Result;
use crossbeam_channel::Sender;
use super::fourcc::spa_to_drm_fourcc;
use super::types::{PortalFormatInfo, PwCtrlEvent, PwDmaBufFrame};
/// PipeWire 捕获线程的上下文数据
///
/// 从主线程传递给 PipeWire 捕获线程的所有必要资源。
/// 该结构体在线程创建时一次性 move 到线程中使用。
struct PwThreadCtx {
frame_tx: Sender<PwDmaBufFrame>,
event_tx: Sender<PwCtrlEvent>,
dropped: Arc<AtomicU64>,
shutdown_read: OwnedFd,
pw_fd: OwnedFd,
node_id: u32,
}
fn pipewire_thread(ctx: PwThreadCtx) {
use pipewire as pw;
use pw::properties::properties;
use pw::spa::param::video::VideoInfoRaw;
use pw::stream::{StreamBox, StreamFlags};
use std::cell::Cell;
use std::rc::Rc;
// 初始化 PipeWire 进程全局库。
//
// pipewire-rs 内部使用 OnceCell 保护 pw::init(),确保只调用一次。
// pw::deinit() 是 unsafe 且要求"进程生命周期内仅调用一次,且所有
// PipeWire 使用已停止"。由于 CapPortal 可被多次创建销毁,此函数
// 不调用 pw::deinit()——进程退出时全局状态由 OS 回收。
pw::init();
let PwThreadCtx {
frame_tx,
event_tx,
dropped,
shutdown_read,
pw_fd,
node_id,
} = ctx;
let mainloop = match pw::main_loop::MainLoopBox::new(None) {
Ok(ml) => ml,
Err(e) => {
if let Err(e) =
event_tx.try_send(PwCtrlEvent::Error(format!("MainLoop::new failed: {e}")))
{
tracing::error!("MainLoop::new failed and error channel also failed: {e}");
}
return;
}
};
let context = match pw::context::ContextBox::new(mainloop.loop_(), None) {
Ok(c) => c,
Err(e) => {
if let Err(e) =
event_tx.try_send(PwCtrlEvent::Error(format!("Context::new failed: {e}")))
{
tracing::error!("Context::new failed and error channel also failed: {e}");
}
return;
}
};
let core = match context.connect_fd(pw_fd, None) {
Ok(c) => c,
Err(e) => {
if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("connect_fd failed: {e}")))
{
tracing::error!("connect_fd failed and error channel also failed: {e}");
}
return;
}
};
// 创建 PipeWire 视频流
// 属性配置:
// - MEDIA_TYPE = "Video": 媒体类型为视频
// - MEDIA_CATEGORY = "Capture": 类别为捕获(而非回放)
// - MEDIA_ROLE = "Screen": 角色为屏幕(用于策略管理)
let stream = match StreamBox::new(
&core,
"wl-webrtc",
properties! {
*pw::keys::MEDIA_TYPE => "Video",
*pw::keys::MEDIA_CATEGORY => "Capture",
*pw::keys::MEDIA_ROLE => "Screen",
*pw::keys::NODE_FORCE_QUANTUM => "512",
},
) {
Ok(s) => s,
Err(e) => {
if let Err(e) =
event_tx.try_send(PwCtrlEvent::Error(format!("Stream::new failed: {e}")))
{
tracing::error!("Stream::new failed and error channel also failed: {e}");
}
return;
}
};
let format_info: Rc<Cell<Option<PortalFormatInfo>>> = Rc::new(Cell::new(None));
let event_tx_state = event_tx.clone();
let _listener = stream
.add_local_listener::<()>()
.state_changed(move |_, _, old, new| {
tracing::info!("PipeWire stream state: {old:?} -> {new:?}");
match new {
pw::stream::StreamState::Error(e) => {
tracing::error!("PipeWire stream error: {e}");
let _ = event_tx_state.try_send(PwCtrlEvent::StreamEnded);
}
pw::stream::StreamState::Unconnected => {
let _ = event_tx_state.try_send(PwCtrlEvent::StreamEnded);
}
pw::stream::StreamState::Paused => {
tracing::warn!("PipeWire stream paused (compositor may be switching content)");
}
pw::stream::StreamState::Streaming => {
tracing::info!("PipeWire stream (re)started");
}
pw::stream::StreamState::Connecting => {}
}
})
// 参数变化回调(格式协商)
// PipeWire 在流格式协商完成后触发此回调
// id 为参数类型,param 包含具体的格式参数(分辨率、像素格式等)
.param_changed({
let format_info = format_info.clone();
let event_tx = event_tx.clone();
move |_, _, id, param| {
// 仅处理 Format 类型的参数变化
let Some(param) = param else { return };
if id != pw::spa::param::ParamType::Format.as_raw() {
return;
}
// 解析视频格式信息(分辨率、像素格式、修饰符等)
let mut info = VideoInfoRaw::new();
if let Err(e) = info.parse(param) {
tracing::warn!("Failed to parse video format: {e}");
return;
}
let width = info.size().width;
let height = info.size().height;
// 将 SPA 视频格式转换为 DRM FourCC 格式标识符
let drm_format = spa_to_drm_fourcc(info.format());
// 获取 DRM 修饰符,描述 GPU buffer 的内存布局(如 tiling 模式)
let modifier = info.modifier();
let framerate = info.framerate();
let max_framerate = info.max_framerate();
// 保存协商后的格式信息,供 process 回调读取
let previous_format = format_info.get();
format_info.set(Some(PortalFormatInfo {
width,
height,
drm_format,
modifier,
}));
if let Some(prev) = previous_format {
if width != prev.width || height != prev.height {
tracing::warn!(
"PipeWire dimensions changed: {}x{} (format renegotiation)",
width,
height
);
let _ = event_tx.try_send(PwCtrlEvent::FormatChanged { width, height });
}
}
tracing::info!(
"PipeWire format negotiated: {width}x{height}, \
drm_format={drm_format:#010x}, modifier={modifier:#x}, \
framerate={}/{}, max_framerate={}/{}",
framerate.num,
framerate.denom,
max_framerate.num,
max_framerate.denom,
);
}
})
// 帧处理回调 —— 这是核心的数据路径
// 每当 PipeWire 有新的帧数据可用时触发
// 关键操作: 从 buffer 中提取 DMA-BUF fddup 后通过 channel 发送给消费者
.process({
let format_info = format_info.clone();
let frame_tx = frame_tx.clone();
move |stream, _| {
// SAFETY: raw_buf ownership invariant — PipeWire's process callback
// contract requires that every buffer acquired via `dequeue_raw_buffer`
// is returned to the queue EXACTLY ONCE via `queue_raw_buffer` before
// the callback returns — on every exit path, success or error. Failure
// to requeue leaks the buffer slot and eventually stalls the stream.
//
// Audit map of this closure (verified 2026-06-28):
// - null raw_buf (dequeue returned NULL) → nothing to requeue, return.
// - null spa_buf / no data / bad fd / null chunk / no format_info /
// invalid dims / dup_fd < 0 → all requeue before early-return.
// - success (try_send Ok / Full / Disconnected) → final requeue at end.
// The fd ownership is independent: dup() creates a fresh fd that lives
// inside PwDmaBufFrame; on try_send error the frame Drops and closes it.
let raw_buf = unsafe { stream.dequeue_raw_buffer() };
if raw_buf.is_null() {
tracing::trace!("process: null raw_buf");
return;
}
// 获取 SPA buffer 结构体,包含数据数组、元数据等
// SAFETY: raw_buf was checked non-null above. `pw_buffer.buffer` is a
// valid raw pointer for the lifetime of raw_buf (PipeWire keeps the
// buffer alive until we queue it back).
let spa_buf = unsafe { (*raw_buf).buffer };
if spa_buf.is_null() {
tracing::trace!("process: null spa_buf");
// SAFETY: raw_buf is the non-null buffer we still own; returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
// 获取 buffer 中的数据项数量和数据指针
// 对于 DMA-BUF 帧,通常只有 1 个数据项(包含 fd)
// SAFETY: spa_buf checked non-null above; `n_datas` is a plain u32 field.
let n_datas = unsafe { (*spa_buf).n_datas };
// SAFETY: same as above; `datas` is a raw pointer field, may be null.
let datas_ptr = unsafe { (*spa_buf).datas };
if n_datas == 0 || datas_ptr.is_null() {
tracing::trace!("process: no data (n_datas={n_datas})");
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
// 从第一个数据项中获取 DMA-BUF 文件描述符
// 通过 libspa 的 Data 包装类型安全地访问 SPA 数据结构
// SAFETY: datas_ptr is non-null and n_datas > 0 (checked above). We cast
// to pw::spa::buffer::Data and take a shared borrow; PipeWire does not
// mutate the data array during a process cycle, so a shared reference
// for the duration of this callback is sound.
let data_ref: &pw::spa::buffer::Data =
unsafe { &*(datas_ptr as *const pw::spa::buffer::Data) };
let fd = data_ref.fd();
if fd < 0 {
tracing::trace!("process: invalid fd={fd}");
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
if data_ref.as_raw().chunk.is_null() {
tracing::trace!("process: null chunk");
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
let chunk = data_ref.chunk();
let offset = chunk.offset() as u64;
let stride = chunk.stride() as u32;
// 从 SPA_META_Header 元数据中提取 PTS (显示时间戳)
// 遍历 buffer 的所有元数据项,查找 Header 类型的元数据
// PTS 可用于音视频同步和帧率控制
// SAFETY: spa_buf is non-null. `metas` is checked for null before
// iteration. We iterate `i in 0..n_metas` reading shared POD fields
// (type_, size, data) — PipeWire keeps the meta array immutable during
// a process cycle. The size guard (`meta.size >= size_of::<spa_meta_header>()`)
// and null-data check before reading ensure we never read past the
// meta's actual extent.
let pts: i64 = unsafe {
let mut pts_val: i64 = 0;
let n_metas = (*spa_buf).n_metas;
let metas = (*spa_buf).metas;
if !metas.is_null() {
for i in 0..n_metas {
let meta = &*metas.add(i as usize);
if meta.type_ == libspa::sys::SPA_META_Header
&& meta.size as usize
>= std::mem::size_of::<libspa::sys::spa_meta_header>()
&& !meta.data.is_null()
{
let header = &*(meta.data as *const libspa::sys::spa_meta_header);
pts_val = header.pts;
break;
}
}
}
pts_val
};
// 验证格式信息已协商完成,且分辨率和格式有效
let Some(fmt) = format_info.get() else {
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
};
let PortalFormatInfo {
width,
height,
drm_format: format,
modifier,
} = fmt;
if width == 0 || height == 0 || format == 0 {
tracing::trace!("process: invalid dimensions {width}x{height} format={format}");
// SAFETY: raw_buf still owned, returning it.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
// 复制 DMA-BUF 文件描述符
// 必须 dup,因为原始 fd 由 PipeWire 管理,我们不能持有它
// dup 后的 fd 由 PwDmaBufFrame 持有,生命周期独立于 PipeWire buffer
// SAFETY: `fd` is the open DMA-BUF fd reported by PipeWire (>= 0 checked
// above). libc::dup is the standard POSIX fd duplication call. The
// original `fd` remains owned by PipeWire (returned with raw_buf later).
let dup_fd = unsafe { libc::dup(fd) };
if dup_fd < 0 {
// SAFETY: raw_buf still owned, returning it. No fd cleanup needed
// because dup() failed and never returned a new fd.
unsafe { stream.queue_raw_buffer(raw_buf) };
return;
}
// 构建帧数据对象,所有必要的帧信息已收集完毕
// SAFETY: `dup_fd` is a freshly-dup'd open file descriptor (>= 0 checked
// above) and we are its sole owner. OwnedFd::from_raw_fd takes ownership
// and will close() it on Drop. The fd's lifecycle is independent of
// raw_buf: whether try_send succeeds (frame moves into the channel) or
// fails (Full/Disconnected — the error payload owns the frame and drops
// it at the end of the match arm), exactly one close() occurs per dup().
let frame_fd = unsafe { OwnedFd::from_raw_fd(dup_fd) };
let frame = PwDmaBufFrame {
fd: frame_fd,
offset,
stride,
modifier,
width,
height,
format,
pts,
};
match frame_tx.try_send(frame) {
Ok(()) => {}
Err(crossbeam_channel::TrySendError::Full(_)) => {
dropped.fetch_add(1, Ordering::Relaxed);
}
Err(crossbeam_channel::TrySendError::Disconnected(_)) => {}
}
// SAFETY: final exactly-once requeue of raw_buf. Every path above
// either returned early with its own requeue, or falls through to here.
unsafe { stream.queue_raw_buffer(raw_buf) };
}
})
.register();
let mut params: [&pw::spa::pod::Pod; 0] = [];
if let Err(e) = stream.connect(
pw::spa::utils::Direction::Input,
Some(node_id),
StreamFlags::AUTOCONNECT | StreamFlags::MAP_BUFFERS,
&mut params,
) {
if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("stream.connect failed: {e}")))
{
tracing::error!("stream.connect failed and error channel also failed: {e}");
}
return;
}
let loop_ = mainloop.loop_();
// Register the shutdown eventfd on the PipeWire loop.
//
// When CapPortal::drop writes to the eventfd, the loop wakes up and
// dispatches this callback on the loop thread. Because the callback
// only fires while mainloop.run() is blocking this thread, mainloop
// is guaranteed alive — eliminating the UAF that existed with the
// previous detached helper thread approach.
// 保存 mainloop 的原始指针,用于在 shutdown 回调中调用 pw_main_loop_quit
// 这是安全的,因为回调只在 mainloop.run() 阻塞期间执行
let mainloop_ptr = mainloop.as_raw_ptr();
let _shutdown_source = loop_.add_io(
shutdown_read,
libspa::support::system::IoFlags::IN,
move |fd| {
// Drain the eventfd so it doesn't re-trigger
let mut buf: u64 = 0;
// SAFETY: `fd` is the registered eventfd owned by the mainloop source; the
// buffer is a stack u64 of 8 bytes matching the count argument. POSIX
// read(2) is the standard fd-read syscall; eventfd semantics require the
// 8-byte buffer.
let _ = unsafe {
libc::read(
fd.as_raw_fd(),
&mut buf as *mut u64 as *mut _,
std::mem::size_of::<u64>(),
)
};
// SAFETY: This callback only executes while mainloop.run() is
// blocking this thread, so mainloop is guaranteed alive.
unsafe { pipewire::sys::pw_main_loop_quit(mainloop_ptr) };
},
);
// 启动 PipeWire 主事件循环
// 此调用会阻塞当前线程,直到 mainloop.quit() 被调用
// quit() 由 shutdown eventfd 的 IO 回调触发
mainloop.run();
// run() returned — _shutdown_source drops first (reverse declaration order),
// which unregisters the callback from the loop. Then mainloop drops.
// No dangling raw pointers are possible.
// PipeWire global state is intentionally not deinitialized here — see pw::init() comment above.
}
pub(super) fn spawn_pipewire_thread(
frame_tx: Sender<PwDmaBufFrame>,
event_tx: Sender<PwCtrlEvent>,
dropped: Arc<AtomicU64>,
shutdown_read: OwnedFd,
pw_fd: OwnedFd,
node_id: u32,
) -> Result<JoinHandle<()>> {
let ctx = PwThreadCtx {
frame_tx,
event_tx,
dropped,
shutdown_read,
pw_fd,
node_id,
};
thread::Builder::new()
.name("pipewire-capture".into())
.spawn(move || pipewire_thread(ctx))
.map_err(|e| anyhow::anyhow!("thread spawn failed: {e}"))
}
+192
View File
@@ -0,0 +1,192 @@
use std::os::fd::OwnedFd;
use anyhow::Result;
use super::logging::log_portal_phase_timeout;
use super::token_fs::{delete_restore_token, load_restore_token, save_restore_token};
use super::types::{PortalPhaseTimeout, PORTAL_SERVICE_TIMEOUT, PORTAL_USER_DIALOG_TIMEOUT};
use super::CapPortal;
impl CapPortal {
/// 通过 XDG Desktop Portal 建立屏幕录制会话
///
/// 与桌面环境的 D-Bus 服务交互,请求用户授权屏幕录制。
/// 流程:
/// 1. 创建 Screencast 代理(D-Bus 代理)
/// 2. 创建 ScreenCast 会话
/// 3. 配置源选择参数(光标模式、显示器源、不持久化会话)
/// 4. 启动录制,获取流信息(包含 PipeWire node_id
/// 5. 打开 PipeWire 远程连接,获取文件描述符
///
/// 返回 (PipeWire fd, node_id),供 PipeWire 线程连接使用
///
/// Wraps `_setup_portal_inner` with token-aware retry: on a `TokenDependent`
/// timeout (phases 3 or 4 with a restore token in use) AND `no_persist ==
/// false`, clears the cached restore token and retries once with
/// `no_persist = true`.
pub(super) async fn setup_portal(no_persist: bool) -> Result<(OwnedFd, u32)> {
match Self::_setup_portal_inner(no_persist, false).await {
Ok(result) => Ok(result),
Err(e) if e.is::<PortalPhaseTimeout>() => {
let inner_err = e.downcast_ref::<PortalPhaseTimeout>().unwrap();
match inner_err {
PortalPhaseTimeout::TokenDependent if !no_persist => {
tracing::warn!(
"Portal timed out during token-using phase. \
Clearing cached restore token and retrying with fresh authorization."
);
delete_restore_token();
Self::_setup_portal_inner(true, true).await
}
_ => Err(e),
}
}
Err(e) => Err(e),
}
}
/// Inner Portal setup with phased timeouts. See `setup_portal` for the
/// retry wrapper.
///
/// `is_retry == true` disables further retry attempts (max 1 retry).
pub(super) async fn _setup_portal_inner(
no_persist: bool,
is_retry: bool,
) -> Result<(OwnedFd, u32)> {
use ashpd::desktop::screencast::{
CursorMode, Screencast, SelectSourcesOptions, SourceType,
};
use ashpd::desktop::PersistMode;
// Phase 1: Screencast proxy (no user interaction).
let proxy = match tokio::time::timeout(PORTAL_SERVICE_TIMEOUT, Screencast::new()).await {
Ok(Ok(p)) => p,
Ok(Err(e)) => return Err(anyhow::anyhow!("Failed to create Screencast proxy: {e}")),
Err(_) => {
log_portal_phase_timeout("creating Screencast proxy", false);
return Err(PortalPhaseTimeout::Service.into());
}
};
// Phase 2: create_session (no user interaction).
let session = match tokio::time::timeout(
PORTAL_SERVICE_TIMEOUT,
proxy.create_session(Default::default()),
)
.await
{
Ok(Ok(s)) => s,
Ok(Err(e)) => return Err(anyhow::anyhow!("Failed to create ScreenCast session: {e}")),
Err(_) => {
log_portal_phase_timeout("creating session", false);
return Err(PortalPhaseTimeout::Service.into());
}
};
let version_supported = proxy.version() >= 4;
let (persist_mode, saved_token) = if !no_persist && version_supported {
let token = load_restore_token();
if token.is_some() {
if is_retry {
tracing::info!("Re-attempting portal session after token clear");
} else {
tracing::info!("Attempting to restore portal session with saved token");
}
}
(PersistMode::ExplicitlyRevoked, token)
} else {
(PersistMode::DoNot, None)
};
let mut options = SelectSourcesOptions::default()
.set_cursor_mode(CursorMode::Embedded)
.set_sources(ashpd::enumflags2::BitFlags::from(SourceType::Monitor))
.set_multiple(false)
.set_persist_mode(persist_mode);
if let Some(ref token) = saved_token {
options = options.set_restore_token(token.as_str());
}
// Phase 3: select_sources — token path is fast (no dialog); fresh
// authorization may pop a dialog.
let token_in_use = saved_token.is_some();
let phase3_timeout = if token_in_use {
PORTAL_SERVICE_TIMEOUT
} else {
PORTAL_USER_DIALOG_TIMEOUT
};
match tokio::time::timeout(phase3_timeout, proxy.select_sources(&session, options)).await {
Ok(Ok(_)) => {}
Ok(Err(e)) => return Err(anyhow::anyhow!("Screen sharing permission denied: {e}")),
Err(_) => {
log_portal_phase_timeout("selecting sources", token_in_use);
return Err(if token_in_use {
PortalPhaseTimeout::TokenDependent
} else {
PortalPhaseTimeout::Service
}
.into());
}
}
// Phase 4: start + response — same dialog-vs-token reasoning as phase 3.
let phase4_timeout = if token_in_use {
PORTAL_SERVICE_TIMEOUT
} else {
PORTAL_USER_DIALOG_TIMEOUT
};
let start_fut = async {
proxy
.start(&session, None, Default::default())
.await?
.response()
};
let response = match tokio::time::timeout(phase4_timeout, start_fut).await {
Ok(Ok(r)) => r,
Ok(Err(e)) => return Err(anyhow::anyhow!("ScreenCast start/response error: {e}")),
Err(_) => {
log_portal_phase_timeout("starting session", token_in_use);
return Err(if token_in_use {
PortalPhaseTimeout::TokenDependent
} else {
PortalPhaseTimeout::Service
}
.into());
}
};
if !no_persist && version_supported {
if let Some(new_token) = response.restore_token() {
save_restore_token(new_token);
}
}
let stream = response
.streams()
.first()
.ok_or_else(|| anyhow::anyhow!("No streams returned from ScreenCast"))?;
let node_id = stream.pipe_wire_node_id();
// Phase 5: open_pipe_wire_remote (no user interaction).
let fd = match tokio::time::timeout(
PORTAL_SERVICE_TIMEOUT,
proxy.open_pipe_wire_remote(&session, Default::default()),
)
.await
{
Ok(Ok(f)) => f,
Ok(Err(e)) => return Err(anyhow::anyhow!("Failed to open PipeWire remote: {e}")),
Err(_) => {
log_portal_phase_timeout("opening PipeWire remote", false);
return Err(PortalPhaseTimeout::Service.into());
}
};
tracing::info!("Portal session established: node_id={node_id}");
Ok((fd, node_id))
}
}
+362
View File
@@ -0,0 +1,362 @@
use std::path::PathBuf;
pub(super) fn token_path() -> Option<PathBuf> {
dirs::cache_dir().map(|base| base.join("wl-webrtc").join("portal-restore-token"))
}
/// Verify that `path` is a directory owned by the current user with no group/other permissions.
/// Rejects symlinks at the path itself (but allows the resolved target to be a real dir).
pub(super) fn verify_secure_dir(path: &std::path::Path) -> bool {
use std::os::unix::fs::{MetadataExt, PermissionsExt};
match std::fs::symlink_metadata(path) {
Ok(meta) => {
if meta.file_type().is_symlink() {
tracing::warn!(
"Token parent dir is a symlink, rejecting: {}",
path.display()
);
return false;
}
// Must be a directory
if !meta.is_dir() {
tracing::warn!("Token parent path is not a directory: {}", path.display());
return false;
}
// Must be owned by current user
// SAFETY: libc::getuid has no preconditions and cannot fail; it simply
// returns the calling process's real user ID.
// SAFETY: libc::getuid has no preconditions and cannot fail.
if meta.uid() != unsafe { libc::getuid() } {
tracing::warn!(
"Token parent dir not owned by current user: {}",
path.display()
);
return false;
}
// No group or other permissions (mode must be 0o700 exactly within the 0o777 mask)
let mode = meta.permissions().mode() & 0o777;
if mode != 0o700 {
tracing::warn!(
"Token parent dir has insecure permissions {:o}, expected 0700: {}",
mode,
path.display()
);
return false;
}
true
}
Err(e) => {
tracing::warn!("Failed to stat token parent dir: {e}");
false
}
}
}
/// Ensure the parent directory exists with restrictive permissions (0o700).
/// Returns false if the directory could not be created or is insecure.
pub(super) fn ensure_secure_parent(parent: &std::path::Path) -> bool {
use std::os::unix::fs::{DirBuilderExt, PermissionsExt};
if parent.exists() {
// Directory exists — try to tighten permissions, then verify.
// set_permissions follows symlinks, which is fine here since
// we verify with symlink_metadata in verify_secure_dir.
if let Err(e) = std::fs::set_permissions(parent, std::fs::Permissions::from_mode(0o700)) {
tracing::warn!("Failed to set directory permissions: {e}");
return false;
}
return verify_secure_dir(parent);
}
// Create with restrictive mode — DirBuilderExt::mode bypasses umask.
let mut builder = std::fs::DirBuilder::new();
builder.recursive(true);
builder.mode(0o700);
if let Err(e) = builder.create(parent) {
tracing::warn!("Failed to create token directory: {e}");
return false;
}
// Verify after creation (belt-and-suspenders)
verify_secure_dir(parent)
}
pub(super) fn load_restore_token() -> Option<String> {
load_restore_token_from(token_path()?)
}
pub(super) fn load_restore_token_from(path: PathBuf) -> Option<String> {
use std::os::unix::fs::{MetadataExt, PermissionsExt};
let meta = match std::fs::symlink_metadata(&path) {
Ok(m) => m,
Err(_) => return None,
};
if meta.file_type().is_symlink() {
tracing::warn!(
"Token file is a symlink, refusing to read: {}",
path.display()
);
return None;
}
if !meta.is_file() {
tracing::warn!("Token path is not a regular file: {}", path.display());
return None;
}
// SAFETY: libc::getuid has no preconditions and cannot fail.
if meta.uid() != unsafe { libc::getuid() } {
tracing::warn!("Token file not owned by current user: {}", path.display());
return None;
}
let mode = meta.permissions().mode() & 0o777;
if mode & 0o077 != 0 {
tracing::warn!(
"Token file has insecure permissions {:o}, refusing to read: {}",
mode,
path.display()
);
return None;
}
let token = std::fs::read_to_string(&path).ok()?;
let trimmed = token.trim().to_string();
if trimmed.is_empty() {
None
} else {
Some(trimmed)
}
}
pub(super) fn save_restore_token(token: &str) {
let Some(path) = token_path() else {
tracing::warn!("No secure cache directory available, skipping token save");
return;
};
save_restore_token_to(token, &path);
}
pub(super) fn delete_restore_token() {
let Some(path) = token_path() else {
return;
};
match std::fs::remove_file(&path) {
Ok(()) => tracing::info!("Deleted stale portal restore token at {}", path.display()),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => tracing::warn!(
"Failed to delete stale restore token at {}: {e}",
path.display()
),
}
}
pub(super) fn save_restore_token_to(token: &str, path: &std::path::Path) {
use std::fs::OpenOptions;
use std::io::Write;
use std::os::unix::fs::OpenOptionsExt;
let Some(parent) = path.parent() else {
tracing::warn!("Token path has no parent directory");
return;
};
if !ensure_secure_parent(parent) {
tracing::warn!("Parent directory is insecure, refusing to save token");
return;
}
// Use a unique temp file to prevent symlink attacks.
// create_new(true) guarantees exclusive creation — fails if file already exists,
// and does NOT follow existing symlinks.
let tmp_path = path.with_extension(format!("{}.tmp", std::process::id()));
let result = (|| -> std::io::Result<()> {
let mut f = OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o600)
.open(&tmp_path)?;
f.write_all(token.as_bytes())?;
f.sync_all()?;
std::fs::rename(&tmp_path, path)?;
Ok(())
})();
match result {
Ok(()) => tracing::info!("Saved portal restore token"),
Err(e) => {
let _ = std::fs::remove_file(&tmp_path);
tracing::warn!("Failed to save restore token: {e}");
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::os::unix::fs::PermissionsExt;
#[test]
fn token_path_never_uses_tmp() {
assert!(token_path().is_some(), "token_path should resolve on Linux");
let path = token_path().unwrap();
assert!(!path.starts_with("/tmp"), "must not fallback to /tmp");
}
#[test]
fn verify_secure_dir_rejects_wrong_permissions() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path();
// 0o700 should pass
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o700)).unwrap();
assert!(verify_secure_dir(path));
// 0o755 should fail
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o755)).unwrap();
assert!(!verify_secure_dir(path));
// 0o777 should fail
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o777)).unwrap();
assert!(!verify_secure_dir(path));
}
#[test]
fn verify_secure_dir_rejects_non_directory() {
let dir = tempfile::tempdir().unwrap();
let file_path = dir.path().join("not-a-dir");
std::fs::write(&file_path, b"test").unwrap();
assert!(!verify_secure_dir(&file_path));
}
#[test]
fn ensure_secure_parent_creates_with_0700() {
let base = tempfile::tempdir().unwrap();
let new_dir = base.path().join("wl-test-new-dir");
assert!(!new_dir.exists());
assert!(ensure_secure_parent(&new_dir));
assert!(new_dir.is_dir());
let meta = std::fs::symlink_metadata(&new_dir).unwrap();
let mode = meta.permissions().mode() & 0o777;
assert_eq!(
mode, 0o700,
"created directory should be 0700, got {mode:o}"
);
}
#[test]
fn ensure_secure_parent_tightens_existing_dir() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path();
// Simulate an existing directory with loose permissions
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o755)).unwrap();
assert!(ensure_secure_parent(path));
let meta = std::fs::symlink_metadata(path).unwrap();
let mode = meta.permissions().mode() & 0o777;
assert_eq!(
mode, 0o700,
"tightened directory should be 0700, got {mode:o}"
);
}
#[test]
fn save_creates_file_with_0600() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
save_restore_token_to("secret-token-123", &token_path);
assert!(token_path.exists());
let meta = std::fs::symlink_metadata(&token_path).unwrap();
let mode = meta.permissions().mode() & 0o777;
assert_eq!(mode, 0o600, "token file should be 0600, got {mode:o}");
assert_eq!(
std::fs::read_to_string(&token_path).unwrap(),
"secret-token-123"
);
}
#[test]
fn load_reads_secure_file() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
// Write a valid 0o600 token file
use std::os::unix::fs::OpenOptionsExt;
let mut f = std::fs::OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o600)
.open(&token_path)
.unwrap();
std::io::Write::write_all(&mut f, b"my-secret\n").unwrap();
let result = load_restore_token_from(token_path);
assert_eq!(result, Some("my-secret".to_string()));
}
#[test]
fn load_rejects_group_readable_file() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
// Write with 0o640 (group readable) — should be rejected
use std::os::unix::fs::OpenOptionsExt;
let mut f = std::fs::OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o640)
.open(&token_path)
.unwrap();
std::io::Write::write_all(&mut f, b"leaked-token\n").unwrap();
let result = load_restore_token_from(token_path);
assert!(result.is_none(), "should reject group-readable token file");
}
#[test]
fn load_rejects_world_readable_file() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
use std::os::unix::fs::OpenOptionsExt;
let mut f = std::fs::OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o604)
.open(&token_path)
.unwrap();
std::io::Write::write_all(&mut f, b"leaked-token\n").unwrap();
let result = load_restore_token_from(token_path);
assert!(result.is_none(), "should reject world-readable token file");
}
#[test]
fn load_rejects_symlink() {
let dir = tempfile::tempdir().unwrap();
let real_path = dir.path().join("real-file");
let link_path = dir.path().join("portal-restore-token");
std::fs::write(&real_path, b"target-content\n").unwrap();
std::os::unix::fs::symlink(&real_path, &link_path).unwrap();
let result = load_restore_token_from(link_path);
assert!(result.is_none(), "should reject symlinked token file");
}
#[test]
fn save_then_load_roundtrip() {
let dir = tempfile::tempdir().unwrap();
let token_path = dir.path().join("portal-restore-token");
save_restore_token_to("roundtrip-token", &token_path);
let loaded = load_restore_token_from(token_path);
assert_eq!(loaded, Some("roundtrip-token".to_string()));
}
}
+79
View File
@@ -0,0 +1,79 @@
use std::os::fd::OwnedFd;
pub(super) const PORTAL_SERVICE_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(5);
pub(super) const PORTAL_USER_DIALOG_TIMEOUT: std::time::Duration =
std::time::Duration::from_secs(30);
/// Classification of Portal phase timeouts to drive retry behavior.
#[derive(Debug)]
pub(super) enum PortalPhaseTimeout {
/// Portal service unresponsive; not retried (user should restart service).
Service,
/// Timed out in token-dependent phase; retried once after clearing token.
TokenDependent,
}
impl std::fmt::Display for PortalPhaseTimeout {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Service => write!(f, "Portal phase timed out (service)"),
Self::TokenDependent => write!(f, "Portal phase timed out (token-dependent)"),
}
}
}
impl std::error::Error for PortalPhaseTimeout {}
/// PipeWire DMA-BUF 帧数据
///
/// 表示从 PipeWire 流中接收到的一帧视频数据。
/// 帧的像素数据存储在 DMA-BUFLinux 的零拷贝 buffer 共享机制)中,
/// 通过文件描述符 (fd) 引用,消费者通过 mmap 或 DRM 导入来访问像素数据。
pub struct PwDmaBufFrame {
/// DMA-BUF 文件描述符,指向 GPU 显存中的帧缓冲区
pub fd: OwnedFd,
/// 帧数据在 DMA-BUF 中的字节偏移量
pub offset: u64,
/// 每行像素的字节跨度(可能大于 width * bpp,因为可能有对齐填充)
pub stride: u32,
/// DRM 格式修饰符,描述 buffer 的内存布局(如线性布局、tiling 等)
pub modifier: u64,
/// 帧宽度(像素)
pub width: u32,
/// 帧高度(像素)
pub height: u32,
/// DRM FourCC 格式标识符(如 BGRA、RGBA 等)
pub format: u32,
/// 显示时间戳 (PTS, Presentation Time Stamp),单位为纳秒
pub pts: i64,
}
/// PipeWire-negotiated video format snapshot, stashed in a `Cell` for cross-callback
/// sharing (format-change callback writes it; process callback reads it). The four
/// fields are the minimal subset of `PwDmaBufFrame`'s metadata that the process
/// callback needs to construct the frame once a buffer arrives.
///
/// `Copy` is required because we store it inside `Cell<Option<PortalFormatInfo>>`;
/// `Cell` requires its contents to be `Copy` (no borrowed interior state).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct PortalFormatInfo {
pub width: u32,
pub height: u32,
/// DRM FourCC format code (e.g. `0x34325258` for XR24 / XRGB8888).
pub drm_format: u32,
/// DRM format modifier describing buffer layout (linear, tiling, etc.).
pub modifier: u64,
}
/// PipeWire 控制事件枚举
///
/// 从 PipeWire 捕获线程发送给消费者的控制事件。
/// 与帧数据分离,通过独立的 channel 传输,确保控制事件不被帧数据淹没。
pub enum PwCtrlEvent {
/// 流已结束(PipeWire 流断开连接或进入错误状态)
StreamEnded,
/// Format/dimensions changed mid-stream
FormatChanged { width: u32, height: u32 },
/// 发生错误,包含错误描述信息
Error(String),
}
+3 -11
View File
@@ -21,9 +21,9 @@ pub struct CapWlrScreencopy {
}
impl CaptureSource for CapWlrScreencopy {
/// Unit type: wlr-screencopy is fully asynchronous — `alloc_frame()`
/// always returns `None`. The frame object is created by Dispatch
/// impls calling `manager.capture_output()`, not by this method.
/// Unit type: wlr-screencopy is fully asynchronous — frame allocation is
/// driven by Dispatch impls calling `manager.capture_output()`, so there
/// is no synchronous `alloc_frame`-style API on this trait.
type Frame = ();
fn new(
@@ -40,14 +40,6 @@ impl CaptureSource for CapWlrScreencopy {
})
}
fn alloc_frame(&mut self) -> Option<Self::Frame> {
// wlr-screencopy is asynchronous: the Dispatch impl creates a new
// ZwlrScreencopyFrameV1 which triggers the buffer allocation flow
// (buffer event → negotiate format → create DMA-BUF). This method
// always returns None.
None
}
fn queue_copy(&mut self, buffer: &WlBuffer, _qh: &QueueHandle<State<Self>>) {
if let Some(frame) = &self.current_frame {
frame.copy(buffer);
+5 -2
View File
@@ -148,6 +148,9 @@ fn run_wlr_screencopy(args: Args) -> Result<()> {
revents: 0,
};
// timeout=0 表示非阻塞,立即返回当前 fd 状态
// SAFETY: `pfd` is a stack-allocated libc::pollfd initialized above with a
// valid wayland_fd and POLLIN events; nfds=1 matches the single-element
// array; timeout=0 is non-blocking. POSIX poll(2) writes revents in place.
let ret = unsafe { libc::poll(&mut pfd, 1, 0) };
tracing::info!(
"Raw poll on wayland fd={wayland_fd}: ret={ret}, revents={}",
@@ -173,7 +176,7 @@ fn run_wlr_screencopy(args: Args) -> Result<()> {
// 注册 SIGINT / SIGTERM 信号用于优雅退出
// signal_hook_mio 将 Unix 信号转换为 fd 可读事件,
// 这样信号也可以通过 epoll 统一监听,不需要单独的信号处理器
let mut signals = signal_hook_mio::v1_0::Signals::new(&[
let mut signals = signal_hook_mio::v1_0::Signals::new([
signal_hook::consts::SIGINT, // Ctrl+C
signal_hook::consts::SIGTERM, // kill 命令默认信号
])?;
@@ -310,7 +313,7 @@ fn run_portal_pipewire(args: Args) -> Result<()> {
// Set up signal handling only (no Wayland fd needed)
// Portal 后端不需要监听 Wayland fd,只需处理 Unix 信号
// 因为帧数据是通过 PipeWire 独立投递的,不走 Wayland 协议
let mut signals = signal_hook_mio::v1_0::Signals::new(&[
let mut signals = signal_hook_mio::v1_0::Signals::new([
signal_hook::consts::SIGINT,
signal_hook::consts::SIGTERM,
])?;
+19
View File
@@ -0,0 +1,19 @@
use wayland_client::protocol::wl_buffer::WlBuffer;
use wayland_client::{Dispatch, Proxy, QueueHandle};
use crate::state::{CaptureSource, State};
impl<S: CaptureSource> Dispatch<WlBuffer, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &WlBuffer,
event: <WlBuffer as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
if let wayland_client::protocol::wl_buffer::Event::Release = event {
tracing::trace!("WlBuffer released");
}
}
}
+115
View File
@@ -0,0 +1,115 @@
use std::mem;
use std::path::PathBuf;
use wayland_client::{Dispatch, Proxy, QueueHandle};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_buffer_params_v1::{
Event as BufferParamsEvent, ZwpLinuxBufferParamsV1,
};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_feedback_v1::{
Event as DmabufFeedbackEvent, ZwpLinuxDmabufFeedbackV1,
};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_v1::{
Event as DmabufEvent, ZwpLinuxDmabufV1,
};
use crate::state::{CaptureSource, EncConstructionStage, InFlightSurface, State};
impl<S: CaptureSource> Dispatch<ZwpLinuxDmabufV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwpLinuxDmabufV1,
event: <ZwpLinuxDmabufV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
DmabufEvent::Format { .. } => {}
DmabufEvent::Modifier { .. } => {}
_ => {}
}
}
}
impl<S: CaptureSource> Dispatch<ZwpLinuxDmabufFeedbackV1, ()> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZwpLinuxDmabufFeedbackV1,
event: <ZwpLinuxDmabufFeedbackV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
DmabufFeedbackEvent::MainDevice { device } => {
if device.len() >= 8 {
let dev_bytes: [u8; 8] = device[..8].try_into().unwrap_or([0u8; 8]);
let dev = u64::from_ne_bytes(dev_bytes);
let minor = ((dev & 0xFF) | ((dev >> 12) & 0xFFFFFF00)) as u32;
let path = PathBuf::from(format!("/dev/dri/renderD{}", minor));
if path.exists() {
tracing::info!(
"Compositor DRM device: {} (dev_t: {})",
path.display(),
dev
);
state.drm_device_from_compositor = Some(path);
} else {
tracing::warn!(
"Compositor reported DRM device {} (dev_t: {}) but path does not exist",
path.display(),
dev
);
}
} else {
tracing::warn!(
"main_device event with unexpected data length: {}",
device.len()
);
}
}
DmabufFeedbackEvent::FormatTable { .. } => {}
DmabufFeedbackEvent::Done => {}
DmabufFeedbackEvent::TrancheDone => {}
DmabufFeedbackEvent::TrancheTargetDevice { .. } => {}
DmabufFeedbackEvent::TrancheFormats { .. } => {}
DmabufFeedbackEvent::TrancheFlags { .. } => {}
_ => {}
}
}
}
impl<S: CaptureSource> Dispatch<ZwpLinuxBufferParamsV1, ()> for State<S> {
fn event(
state: &mut Self,
proxy: &ZwpLinuxBufferParamsV1,
event: <ZwpLinuxBufferParamsV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
BufferParamsEvent::Created { .. } => {
tracing::debug!("DMA-BUF buffer created");
}
BufferParamsEvent::Failed => {
tracing::error!("DMA-BUF buffer creation failed");
let taken = mem::replace(&mut state.in_flight_surface, InFlightSurface::None);
match taken {
InFlightSurface::CopyQueued { buffer, frame, .. } => {
drop(buffer);
if let EncConstructionStage::Streaming { cap, .. } = &mut state.stage {
cap.on_done_with_frame(frame);
}
}
other => {
state.in_flight_surface = other;
}
}
proxy.destroy();
state.errored = true;
}
_ => {}
}
}
}
+6
View File
@@ -0,0 +1,6 @@
mod buffer;
mod dmabuf;
mod output_mgr;
mod registry;
mod screencopy;
mod wl_output;
+109
View File
@@ -0,0 +1,109 @@
use wayland_client::{event_created_child, Dispatch, Proxy, QueueHandle};
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_manager_v1::ZxdgOutputManagerV1;
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_head_v1::{
self, Event as WlrHeadEvent, ZwlrOutputHeadV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_manager_v1::{
self, Event as WlrOutputManagerEvent, ZwlrOutputManagerV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_mode_v1::ZwlrOutputModeV1;
use crate::state::{CaptureSource, EncConstructionStage, State, WlrHeadInfo};
impl<S: CaptureSource> Dispatch<ZxdgOutputManagerV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZxdgOutputManagerV1,
_event: <ZxdgOutputManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
impl<S: CaptureSource> Dispatch<ZwlrOutputManagerV1, ()> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZwlrOutputManagerV1,
event: <ZwlrOutputManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
WlrOutputManagerEvent::Head { head } => {
let _head: ZwlrOutputHeadV1 = head;
tracing::debug!("wlr output head advertised");
}
WlrOutputManagerEvent::Done { .. } => {
if let EncConstructionStage::ProbingOutputs {
wlr_manager_done,
outputs,
..
} = &mut state.stage
{
*wlr_manager_done = true;
let count = outputs.len();
for idx in 0..count {
state.try_finalize_output(idx);
}
}
}
WlrOutputManagerEvent::Finished => {
tracing::warn!("zwlr_output_manager_v1::Finished received during probing");
}
_ => {}
}
}
event_created_child!(State<S>, ZwlrOutputManagerV1, [
zwlr_output_manager_v1::EVT_HEAD_OPCODE => (ZwlrOutputHeadV1, ()),
]);
}
impl<S: CaptureSource> Dispatch<ZwlrOutputHeadV1, ()> for State<S> {
fn event(
state: &mut Self,
proxy: &ZwlrOutputHeadV1,
event: <ZwlrOutputHeadV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
WlrHeadEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs {
wlr_heads,
wlr_head_proxy_to_name,
..
} = &mut state.stage
{
wlr_heads.entry(name.clone()).or_insert(WlrHeadInfo {});
wlr_head_proxy_to_name.insert(proxy.id(), name);
}
}
WlrHeadEvent::Position { .. } => {}
WlrHeadEvent::Finished => {
tracing::debug!("zwlr_output_head_v1::Finished received");
}
_ => {}
}
}
event_created_child!(State<S>, ZwlrOutputHeadV1, [
zwlr_output_head_v1::EVT_MODE_OPCODE => (ZwlrOutputModeV1, ()),
]);
}
impl<S: CaptureSource> Dispatch<ZwlrOutputModeV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwlrOutputModeV1,
_event: <ZwlrOutputModeV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
+116
View File
@@ -0,0 +1,116 @@
use wayland_client::globals::GlobalListContents;
use wayland_client::protocol::wl_output::WlOutput;
use wayland_client::protocol::wl_registry::WlRegistry;
use wayland_client::{Dispatch, QueueHandle};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_v1::ZwpLinuxDmabufV1;
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_manager_v1::ZxdgOutputManagerV1;
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_manager_v1::ZwlrOutputManagerV1;
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_manager_v1::ZwlrScreencopyManagerV1;
use crate::state::{CaptureSource, EncConstructionStage, OutputId, PartialOutputInfo, State};
impl<S: CaptureSource> Dispatch<WlRegistry, GlobalListContents> for State<S> {
fn event(
state: &mut Self,
registry: &WlRegistry,
event: wayland_client::protocol::wl_registry::Event,
_data: &GlobalListContents,
_conn: &wayland_client::Connection,
qhandle: &QueueHandle<State<S>>,
) {
use wayland_client::protocol::wl_registry::Event as RegistryEvent;
match event {
RegistryEvent::Global {
name,
interface,
version,
} => match interface.as_str() {
"zwlr_screencopy_manager_v1" => {
let v = version.min(3);
tracing::debug!("Binding zwlr_screencopy_manager_v1 v{v} (name={name})");
let mgr: ZwlrScreencopyManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
screencopy_manager, ..
} = &mut state.stage
{
*screencopy_manager = Some(mgr);
}
}
"zwp_linux_dmabuf_v1" => {
let v = version.min(4);
tracing::debug!("Binding zwp_linux_dmabuf_v1 v{v} (name={name})");
let proxy: ZwpLinuxDmabufV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
dmabuf,
dmabuf_feedback,
..
} = &mut state.stage
{
*dmabuf = Some(proxy.clone());
if v >= 4 {
let feedback = proxy.get_default_feedback(qhandle, ());
*dmabuf_feedback = Some(feedback);
}
}
}
"wl_output" => {
let v = version.min(4);
tracing::debug!("Binding wl_output v{v} (name={name})");
let output: WlOutput = registry.bind(name, v, qhandle, OutputId(name));
if let EncConstructionStage::ProbingOutputs {
outputs,
bound_outputs,
output_names,
xdg_output_manager,
..
} = &mut state.stage
{
outputs.push(PartialOutputInfo::default());
bound_outputs.push(output.clone());
output_names.push(name);
if let Some(xdg_mgr) = xdg_output_manager {
let output_id = OutputId(name);
xdg_mgr.get_xdg_output(&output, qhandle, output_id);
}
}
}
"zxdg_output_manager_v1" => {
let v = version.min(3);
tracing::debug!("Binding zxdg_output_manager_v1 v{v} (name={name})");
let xdg_mgr: ZxdgOutputManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
bound_outputs,
xdg_output_manager,
output_names,
..
} = &mut state.stage
{
for (i, output) in bound_outputs.iter().enumerate() {
let oname = output_names.get(i).copied().unwrap_or(0);
let output_id = OutputId(oname);
xdg_mgr.get_xdg_output(output, qhandle, output_id);
}
*xdg_output_manager = Some(xdg_mgr);
}
}
"zwlr_output_manager_v1" => {
let v = version.min(4);
tracing::debug!("Binding zwlr_output_manager_v1 v{v} (name={name})");
let mgr: ZwlrOutputManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
wlr_output_manager, ..
} = &mut state.stage
{
*wlr_output_manager = Some(mgr);
}
}
_ => {}
},
RegistryEvent::GlobalRemove { name } => {
tracing::debug!("Global removed: name={name}");
}
_ => {}
}
}
}
+93
View File
@@ -0,0 +1,93 @@
use wayland_client::{Dispatch, Proxy, QueueHandle};
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_frame_v1::{
Event as ScreencopyFrameEvent, ZwlrScreencopyFrameV1,
};
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_manager_v1::ZwlrScreencopyManagerV1;
use crate::cap_wlr_screencopy::CapWlrScreencopy;
use crate::state::{EncConstructionStage, InFlightSurface, State};
impl Dispatch<ZwlrScreencopyFrameV1, ()> for State<CapWlrScreencopy> {
fn event(
state: &mut Self,
proxy: &ZwlrScreencopyFrameV1,
event: <ZwlrScreencopyFrameV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<CapWlrScreencopy>>,
) {
match event {
// SHM buffer offer — in v3 the compositor enumerates supported buffer
// types (buffer and/or linux_dmabuf) before buffer_done. We only
// support DMA-BUF, so just log and wait for linux_dmabuf / buffer_done.
ScreencopyFrameEvent::Buffer { .. } => {
tracing::debug!("Received SHM Buffer offer — only DMA-BUF capture is supported");
}
ScreencopyFrameEvent::LinuxDmabuf {
format,
width,
height,
} => {
tracing::debug!("Screencopy LinuxDmabuf: format={format}, {width}x{height}");
if !matches!(state.in_flight_surface, InFlightSurface::AllocQueued) {
tracing::warn!("Received LinuxDmabuf while no frame allocation was queued");
return;
}
if matches!(state.stage, EncConstructionStage::EverythingButFmt { .. }) {
state.negotiate_format(format, width, height);
if state.errored {
return;
}
}
if let EncConstructionStage::Streaming { cap, .. } = &mut state.stage {
cap.current_frame = Some(proxy.clone());
}
state.on_frame_allocd((), format, width, height);
}
// v3 terminal event: all buffer offers have been enumerated.
// If still AllocQueued, the compositor never sent linux_dmabuf —
// DMA-BUF screencopy is unsupported, so we must error out.
ScreencopyFrameEvent::BufferDone => {
if matches!(state.in_flight_surface, InFlightSurface::AllocQueued) {
tracing::error!(
"Compositor did not offer DMA-BUF screencopy (only SHM); \
DMA-BUF capture is required"
);
state.in_flight_surface = InFlightSurface::None;
proxy.destroy();
state.errored = true;
}
}
ScreencopyFrameEvent::Ready {
tv_sec_hi,
tv_sec_lo,
tv_nsec,
} => {
let tv_sec = (tv_sec_hi as u64) << 32 | tv_sec_lo as u64;
let tv_usec = tv_nsec / 1000;
tracing::trace!("Screencopy ready: tv_sec={tv_sec}, tv_usec={tv_usec}");
state.on_copy_complete(tv_sec, tv_usec);
}
ScreencopyFrameEvent::Failed => {
tracing::error!("Screencopy frame failed");
state.on_copy_fail();
}
ScreencopyFrameEvent::Damage { .. } => {}
_ => {}
}
}
}
impl<S: crate::state::CaptureSource> Dispatch<ZwlrScreencopyManagerV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwlrScreencopyManagerV1,
_event: <ZwlrScreencopyManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
+133
View File
@@ -0,0 +1,133 @@
use wayland_client::protocol::wl_output::WlOutput;
use wayland_client::{Dispatch, Proxy, QueueHandle};
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_v1::{
Event as XdgOutputEvent, ZxdgOutputV1,
};
use crate::state::{CaptureSource, EncConstructionStage, OutputId, State, Transform};
impl<S: CaptureSource> Dispatch<WlOutput, OutputId> for State<S> {
fn event(
state: &mut Self,
_proxy: &WlOutput,
event: wayland_client::protocol::wl_output::Event,
data: &OutputId,
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
use wayland_client::protocol::wl_output::Event as OutputEvent;
use wayland_client::protocol::wl_output::Mode as WlMode;
use wayland_client::protocol::wl_output::Transform as WlTransform;
let OutputId(target_name) = data;
let idx = match &state.stage {
EncConstructionStage::ProbingOutputs { output_names, .. } => {
output_names.iter().position(|&n| n == *target_name)
}
_ => None,
};
let idx = match idx {
Some(i) => i,
None => return,
};
match event {
OutputEvent::Geometry { transform, .. } => {
let t = match transform {
wayland_client::WEnum::Value(WlTransform::Normal) => Transform::Normal,
wayland_client::WEnum::Value(WlTransform::_90) => Transform::Normal90,
wayland_client::WEnum::Value(WlTransform::_180) => Transform::Normal180,
wayland_client::WEnum::Value(WlTransform::_270) => Transform::Normal270,
wayland_client::WEnum::Value(WlTransform::Flipped) => Transform::Flipped,
wayland_client::WEnum::Value(WlTransform::Flipped90) => Transform::Flipped90,
wayland_client::WEnum::Value(WlTransform::Flipped180) => Transform::Flipped180,
wayland_client::WEnum::Value(WlTransform::Flipped270) => Transform::Flipped270,
_ => Transform::Normal,
};
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.transform = Some(t);
}
}
}
OutputEvent::Mode {
width,
height,
flags,
..
} => {
let is_current = matches!(flags, wayland_client::WEnum::Value(WlMode::Current));
if is_current {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.mode_size = Some((width, height));
}
}
}
}
OutputEvent::Done => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.done_count += 1;
if info.done_count >= 1 {
state.try_finalize_output(idx);
}
}
}
}
OutputEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.wl_name = Some(name);
}
}
}
_ => {}
}
}
}
impl<S: CaptureSource> Dispatch<ZxdgOutputV1, OutputId> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZxdgOutputV1,
event: <ZxdgOutputV1 as Proxy>::Event,
data: &OutputId,
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
let target_name = data.0;
let idx = match &state.stage {
EncConstructionStage::ProbingOutputs { output_names, .. } => {
output_names.iter().position(|&n| n == target_name)
}
_ => None,
};
let idx = match idx {
Some(i) => i,
None => return,
};
match event {
XdgOutputEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.name = Some(name);
}
}
}
XdgOutputEvent::LogicalSize { .. } => {}
XdgOutputEvent::Done => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.done_count += 1;
if info.done_count >= 1 {
state.try_finalize_output(idx);
}
}
}
}
_ => {}
}
}
}
+57 -705
View File
@@ -9,34 +9,16 @@ use std::time::Instant;
use anyhow::Result;
use wayland_client::backend::ObjectId;
use wayland_client::globals::{GlobalList, GlobalListContents};
use wayland_client::globals::GlobalList;
use wayland_client::protocol::wl_buffer::WlBuffer;
use wayland_client::protocol::wl_output::WlOutput;
use wayland_client::protocol::wl_registry::WlRegistry;
use wayland_client::{event_created_child, Dispatch, Proxy, QueueHandle};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_buffer_params_v1::{
Event as BufferParamsEvent, Flags as BufferParamsFlags, ZwpLinuxBufferParamsV1,
};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_feedback_v1::{
Event as DmabufFeedbackEvent, ZwpLinuxDmabufFeedbackV1,
};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_v1::{
Event as DmabufEvent, ZwpLinuxDmabufV1,
};
use wayland_client::{Dispatch, QueueHandle};
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_buffer_params_v1::Flags as BufferParamsFlags;
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_feedback_v1::ZwpLinuxDmabufFeedbackV1;
use wayland_protocols::wp::linux_dmabuf::zv1::client::zwp_linux_dmabuf_v1::ZwpLinuxDmabufV1;
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_manager_v1::ZxdgOutputManagerV1;
use wayland_protocols::xdg::xdg_output::zv1::client::zxdg_output_v1::{
Event as XdgOutputEvent, ZxdgOutputV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_head_v1::{
self, Event as WlrHeadEvent, ZwlrOutputHeadV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_manager_v1::{
self, Event as WlrOutputManagerEvent, ZwlrOutputManagerV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_mode_v1::ZwlrOutputModeV1;
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_frame_v1::{
Event as ScreencopyFrameEvent, ZwlrScreencopyFrameV1,
};
use wayland_protocols_wlr::output_management::v1::client::zwlr_output_manager_v1::ZwlrOutputManagerV1;
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_frame_v1::ZwlrScreencopyFrameV1;
use wayland_protocols_wlr::screencopy::v1::client::zwlr_screencopy_manager_v1::ZwlrScreencopyManagerV1;
use ffmpeg_next as ff;
@@ -44,12 +26,13 @@ use ffmpeg_next::ffi;
use crate::args::Args;
use crate::avhw::{AvHwDevCtx, EncState, EncodedH264Frame, SwEncState};
use crate::cap_wlr_screencopy::CapWlrScreencopy;
use crate::fps_limit::FpsLimit;
use crate::stats::{FrameTimings, PipelineStats};
use crate::transform::{transpose_if_transform_transposed, Transform};
use crate::webrtc::WebRtcState;
mod dispatch;
// ---------------------------------------------------------------------------
// CaptureSource trait
// ---------------------------------------------------------------------------
@@ -65,8 +48,6 @@ pub trait CaptureSource: Sized + 'static {
qh: &QueueHandle<State<Self>>,
) -> Result<Self>;
fn alloc_frame(&mut self) -> Option<Self::Frame>;
fn queue_copy(&mut self, buffer: &WlBuffer, qh: &QueueHandle<State<Self>>);
fn on_done_with_frame(&mut self, frame: Self::Frame);
@@ -79,40 +60,24 @@ pub trait CaptureSource: Sized + 'static {
pub struct OutputInfo {
pub name: String,
pub transform: Transform,
pub physical_size: (i32, i32),
pub logical_position: (i32, i32),
}
#[derive(Default)]
pub struct PartialOutputInfo {
pub name: Option<String>,
/// Name from wl_output::Name (v4) — used to match wlr-output-management heads
pub wl_name: Option<String>,
pub transform: Option<Transform>,
pub physical_size: Option<(i32, i32)>,
pub logical_position: Option<(i32, i32)>,
// Pixel dimensions from Mode event — preparatory for Phase 2 resolution logic
pub mode_size: Option<(i32, i32)>,
pub done_count: u32,
}
impl Default for PartialOutputInfo {
fn default() -> Self {
Self {
name: None,
wl_name: None,
transform: None,
physical_size: None,
logical_position: None,
mode_size: None,
done_count: 0,
}
}
}
/// Stores head info from wlr-output-management for name-based matching with wl_output.
struct WlrHeadInfo {
position: Option<(i32, i32)>,
}
/// Marker for wlr-output-management heads seen during probing; tracked by name
/// in `EncConstructionStage::ProbingOutputs.wlr_heads`.
// `pub(crate)` (not module-private): exposed via `EncConstructionStage::ProbingOutputs.wlr_heads`
// which is reached from main.rs during the wlr-screencopy probing loop.
pub(crate) struct WlrHeadInfo {}
/// User data for XdgOutput dispatch to identify which WlOutput it belongs to.
pub struct OutputId(pub u32);
@@ -138,7 +103,10 @@ impl StreamingEncoder {
}
}
fn encode_frame(&mut self, hw_frame: &ffmpeg_next::frame::Video) -> anyhow::Result<()> {
fn encode_frame(
&mut self,
hw_frame: &ffmpeg_next::frame::Video,
) -> anyhow::Result<crate::avhw::EncodeStages> {
match self {
StreamingEncoder::Mp4(enc) => enc.encode_frame(hw_frame),
StreamingEncoder::WebRtc(enc) => enc.encode_frame(hw_frame),
@@ -156,8 +124,14 @@ impl StreamingEncoder {
// ---------------------------------------------------------------------------
// EncConstructionStage
// ---------------------------------------------------------------------------
//
// `pub(crate)` (not `pub`): this enum leaks the private `WlrHeadInfo` type via
// its `wlr_heads` field, and the construction-stage state machine is an
// internal implementation detail. Crate-internal consumers (main.rs) get there
// via `crate::state::`; there is no need to expose this across the crate
// boundary. See Oracle audit 2026-06-28.
pub enum EncConstructionStage<S: CaptureSource> {
pub(crate) enum EncConstructionStage<S: CaptureSource> {
ProbingOutputs {
outputs: Vec<PartialOutputInfo>,
bound_outputs: Vec<WlOutput>,
@@ -180,7 +154,6 @@ pub enum EncConstructionStage<S: CaptureSource> {
dmabuf: ZwpLinuxDmabufV1,
},
Streaming {
output_info: OutputInfo,
output: WlOutput,
enc: StreamingEncoder,
cap: S,
@@ -197,10 +170,12 @@ pub enum EncConstructionStage<S: CaptureSource> {
pub enum InFlightSurface<S: CaptureSource> {
None,
AllocQueued,
Allocd(S::Frame),
CopyQueued {
surface: ff::frame::Video,
drm_map: ff::ffi::AVDRMFrameDescriptor,
// Boxed: AVDRMFrameDescriptor is ~592 bytes (4 objects + 4 layers),
// which would balloon every InFlightSurface variant via enum alignment.
// The box shrinks the enum to ~32 bytes regardless of variant.
drm_map: Box<ff::ffi::AVDRMFrameDescriptor>,
frame: S::Frame,
buffer: WlBuffer,
},
@@ -211,9 +186,8 @@ pub enum InFlightSurface<S: CaptureSource> {
// ---------------------------------------------------------------------------
pub struct State<S: CaptureSource> {
pub stage: EncConstructionStage<S>,
pub(crate) stage: EncConstructionStage<S>,
pub in_flight_surface: InFlightSurface<S>,
pub starting_timestamp: Option<i64>,
pub stats_start_time: Option<Instant>,
pub stats_last_time: Option<Instant>,
pub stats_frames: u64,
@@ -306,7 +280,6 @@ impl<S: CaptureSource> State<S> {
wlr_head_proxy_to_name: HashMap::new(),
},
in_flight_surface: InFlightSurface::None,
starting_timestamp: None,
stats_start_time: None,
stats_last_time: None,
stats_frames: 0,
@@ -483,7 +456,6 @@ impl<S: CaptureSource> State<S> {
pub fn on_frame_allocd(&mut self, frame: S::Frame, format: u32, width: u32, height: u32) {
let (frames_rgb_ctx, dmabuf, cap) = match &mut self.stage {
EncConstructionStage::Streaming {
output_info: _,
output: _,
enc,
dmabuf,
@@ -512,6 +484,10 @@ impl<S: CaptureSource> State<S> {
unsafe {
(*map_frame.as_mut_ptr()).format = ffi::AVPixelFormat::AV_PIX_FMT_DRM_PRIME as i32;
}
// SAFETY: map_frame and surface are valid, owned AVFrame pointers from
// av_hwframe_get/surface.alloc above. AV_HWFRAME_MAP_READ flag (0 here)
// requests a read-only mapping. The DRM_PRIME format set above instructs
// FFmpeg to populate data[0] with an AVDRMFrameDescriptor on success.
let ret = unsafe { ffi::av_hwframe_map(map_frame.as_mut_ptr(), surface.as_ptr(), 0) };
if ret < 0 {
tracing::error!("av_hwframe_map failed: {}", crate::avhw::ff_err(ret));
@@ -571,7 +547,7 @@ impl<S: CaptureSource> State<S> {
);
self.in_flight_surface = InFlightSurface::CopyQueued {
surface,
drm_map: desc,
drm_map: Box::new(desc),
frame,
buffer: wl_buffer,
};
@@ -629,15 +605,22 @@ impl<S: CaptureSource> State<S> {
};
if should_encode {
let encode_start = Instant::now();
if let Err(e) = enc.encode_frame(&surface) {
tracing::error!("encode_frame failed: {}", e);
self.errored = true;
match enc.encode_frame(&surface) {
Ok(stages) => {
let encode_elapsed = encode_start.elapsed().as_micros() as u64;
self.stats.record_encode(&FrameTimings {
scale_us: stages.scale_us,
transfer_us: stages.transfer_us,
encode_us: stages.encode_us,
total_us: encode_elapsed,
..Default::default()
});
}
Err(e) => {
tracing::error!("encode_frame failed: {}", e);
self.errored = true;
}
}
let encode_elapsed = encode_start.elapsed().as_micros() as u64;
self.stats.record_encode(&FrameTimings {
total_us: encode_elapsed,
..Default::default()
});
}
self.stats_frames += 1;
if let Some(last) = self.stats_last_time {
@@ -706,9 +689,7 @@ impl<S: CaptureSource> State<S> {
continue;
}
count += 1;
if let Err(e) = wrtc
.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks)
{
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks) {
tracing::debug!("WebRTC write frame error: {e}");
}
self.stats.record_send(0.0, None);
@@ -822,7 +803,6 @@ impl<S: CaptureSource> State<S> {
bitrate
);
self.stage = EncConstructionStage::Streaming {
output_info,
output,
enc,
cap,
@@ -832,22 +812,6 @@ impl<S: CaptureSource> State<S> {
}
fn try_finalize_output(&mut self, _idx: usize) -> bool {
// Merge wlr head position info into outputs (needed for niri path)
if let EncConstructionStage::ProbingOutputs {
outputs, wlr_heads, ..
} = &mut self.stage
{
for info in outputs.iter_mut() {
if info.logical_position.is_none() {
if let Some(ref wl_name) = info.wl_name {
if let Some(head_info) = wlr_heads.get(wl_name) {
info.logical_position = head_info.position;
}
}
}
}
}
let (target_idx, output_count) = match &self.stage {
EncConstructionStage::ProbingOutputs {
outputs,
@@ -890,24 +854,20 @@ impl<S: CaptureSource> State<S> {
Some(i) => {
let info = &outputs[i];
if has_xdg {
// xdg-output path (Sway/Hyprland) — strict checks
// done_count >= 2 implies physical_size and logical_position
// already arrived (Wayland: Geometry/Mode/Position fire before Done).
if info.done_count < 2
|| info.name.is_none()
|| info.transform.is_none()
|| info.physical_size.is_none()
|| info.logical_position.is_none()
{
return false;
}
} else {
// wlr-output-management path (niri) — relaxed checks
if info.done_count < 1 || !wlr_manager_done {
// done_count >= 1 implies transform arrived (Geometry precedes Done).
if info.done_count < 1 || !wlr_manager_done || info.transform.is_none()
{
return false;
}
if info.transform.is_none() || info.physical_size.is_none() {
return false;
}
// name and logical_position can use defaults
}
(i, output_count)
}
@@ -978,8 +938,6 @@ impl<S: CaptureSource> State<S> {
.or(info.wl_name.clone())
.unwrap_or_else(|| format!("output-{}", output_names[target_idx])),
transform: info.transform.unwrap(),
physical_size: info.physical_size.unwrap(),
logical_position: info.logical_position.unwrap_or((0, 0)),
};
let output = bound_outputs[target_idx].clone();
@@ -1039,609 +997,3 @@ impl<S: CaptureSource> State<S> {
true
}
}
// ---------------------------------------------------------------------------
// Dispatch<WlRegistry, GlobalListContents>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<WlRegistry, GlobalListContents> for State<S> {
fn event(
state: &mut Self,
registry: &WlRegistry,
event: wayland_client::protocol::wl_registry::Event,
_data: &GlobalListContents,
_conn: &wayland_client::Connection,
qhandle: &QueueHandle<State<S>>,
) {
use wayland_client::protocol::wl_registry::Event as RegistryEvent;
match event {
RegistryEvent::Global {
name,
interface,
version,
} => match interface.as_str() {
"zwlr_screencopy_manager_v1" => {
let v = version.min(3);
tracing::debug!("Binding zwlr_screencopy_manager_v1 v{v} (name={name})");
let mgr: ZwlrScreencopyManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
screencopy_manager, ..
} = &mut state.stage
{
*screencopy_manager = Some(mgr);
}
}
"zwp_linux_dmabuf_v1" => {
let v = version.min(4);
tracing::debug!("Binding zwp_linux_dmabuf_v1 v{v} (name={name})");
let proxy: ZwpLinuxDmabufV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
dmabuf,
dmabuf_feedback,
..
} = &mut state.stage
{
*dmabuf = Some(proxy.clone());
if v >= 4 {
let feedback = proxy.get_default_feedback(qhandle, ());
*dmabuf_feedback = Some(feedback);
}
}
}
"wl_output" => {
let v = version.min(4);
tracing::debug!("Binding wl_output v{v} (name={name})");
let output: WlOutput = registry.bind(name, v, qhandle, OutputId(name));
if let EncConstructionStage::ProbingOutputs {
outputs,
bound_outputs,
output_names,
xdg_output_manager,
..
} = &mut state.stage
{
outputs.push(PartialOutputInfo::default());
bound_outputs.push(output.clone());
output_names.push(name);
if let Some(xdg_mgr) = xdg_output_manager {
let output_id = OutputId(name);
xdg_mgr.get_xdg_output(&output, qhandle, output_id);
}
}
}
"zxdg_output_manager_v1" => {
let v = version.min(3);
tracing::debug!("Binding zxdg_output_manager_v1 v{v} (name={name})");
let xdg_mgr: ZxdgOutputManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
bound_outputs,
xdg_output_manager,
output_names,
..
} = &mut state.stage
{
for (i, output) in bound_outputs.iter().enumerate() {
let oname = output_names.get(i).copied().unwrap_or(0);
let output_id = OutputId(oname);
xdg_mgr.get_xdg_output(output, qhandle, output_id);
}
*xdg_output_manager = Some(xdg_mgr);
}
}
"zwlr_output_manager_v1" => {
let v = version.min(4);
tracing::debug!("Binding zwlr_output_manager_v1 v{v} (name={name})");
let mgr: ZwlrOutputManagerV1 = registry.bind(name, v, qhandle, ());
if let EncConstructionStage::ProbingOutputs {
wlr_output_manager, ..
} = &mut state.stage
{
*wlr_output_manager = Some(mgr);
}
}
_ => {}
},
RegistryEvent::GlobalRemove { name } => {
tracing::debug!("Global removed: name={name}");
}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<WlOutput, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<WlOutput, OutputId> for State<S> {
fn event(
state: &mut Self,
_proxy: &WlOutput,
event: wayland_client::protocol::wl_output::Event,
data: &OutputId,
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
use wayland_client::protocol::wl_output::Event as OutputEvent;
use wayland_client::protocol::wl_output::Mode as WlMode;
use wayland_client::protocol::wl_output::Transform as WlTransform;
let OutputId(target_name) = data;
let idx = match &state.stage {
EncConstructionStage::ProbingOutputs { output_names, .. } => {
output_names.iter().position(|&n| n == *target_name)
}
_ => None,
};
let idx = match idx {
Some(i) => i,
None => return,
};
match event {
OutputEvent::Geometry {
transform,
physical_width,
physical_height,
..
} => {
let t = match transform {
wayland_client::WEnum::Value(WlTransform::Normal) => Transform::Normal,
wayland_client::WEnum::Value(WlTransform::_90) => Transform::Normal90,
wayland_client::WEnum::Value(WlTransform::_180) => Transform::Normal180,
wayland_client::WEnum::Value(WlTransform::_270) => Transform::Normal270,
wayland_client::WEnum::Value(WlTransform::Flipped) => Transform::Flipped,
wayland_client::WEnum::Value(WlTransform::Flipped90) => Transform::Flipped90,
wayland_client::WEnum::Value(WlTransform::Flipped180) => Transform::Flipped180,
wayland_client::WEnum::Value(WlTransform::Flipped270) => Transform::Flipped270,
_ => Transform::Normal,
};
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.transform = Some(t);
info.physical_size = Some((physical_width, physical_height));
}
}
}
OutputEvent::Mode {
width,
height,
flags,
..
} => {
let is_current = matches!(flags, wayland_client::WEnum::Value(WlMode::Current));
if is_current {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.mode_size = Some((width, height));
}
}
}
}
OutputEvent::Done => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.done_count += 1;
if info.done_count >= 1 {
state.try_finalize_output(idx);
}
}
}
}
OutputEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.wl_name = Some(name);
}
}
}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZxdgOutputV1, OutputId>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZxdgOutputV1, OutputId> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZxdgOutputV1,
event: <ZxdgOutputV1 as Proxy>::Event,
data: &OutputId,
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
let target_name = data.0;
let idx = match &state.stage {
EncConstructionStage::ProbingOutputs { output_names, .. } => {
output_names.iter().position(|&n| n == target_name)
}
_ => None,
};
let idx = match idx {
Some(i) => i,
None => return,
};
match event {
XdgOutputEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.name = Some(name);
}
}
}
XdgOutputEvent::LogicalPosition { x, y } => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.logical_position = Some((x, y));
}
}
}
XdgOutputEvent::LogicalSize { .. } => {}
XdgOutputEvent::Done => {
if let EncConstructionStage::ProbingOutputs { outputs, .. } = &mut state.stage {
if let Some(info) = outputs.get_mut(idx) {
info.done_count += 1;
if info.done_count >= 1 {
state.try_finalize_output(idx);
}
}
}
}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwpLinuxDmabufV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwpLinuxDmabufV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwpLinuxDmabufV1,
event: <ZwpLinuxDmabufV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
DmabufEvent::Format { .. } => {}
DmabufEvent::Modifier { .. } => {}
_ => {}
}
}
}
impl<S: CaptureSource> Dispatch<ZwpLinuxDmabufFeedbackV1, ()> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZwpLinuxDmabufFeedbackV1,
event: <ZwpLinuxDmabufFeedbackV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
DmabufFeedbackEvent::MainDevice { device } => {
if device.len() >= 8 {
let dev_bytes: [u8; 8] = device[..8].try_into().unwrap_or([0u8; 8]);
let dev = u64::from_ne_bytes(dev_bytes);
let minor = ((dev & 0xFF) | ((dev >> 12) & 0xFFFFFF00)) as u32;
let path = PathBuf::from(format!("/dev/dri/renderD{}", minor));
if path.exists() {
tracing::info!(
"Compositor DRM device: {} (dev_t: {})",
path.display(),
dev
);
state.drm_device_from_compositor = Some(path);
} else {
tracing::warn!(
"Compositor reported DRM device {} (dev_t: {}) but path does not exist",
path.display(),
dev
);
}
} else {
tracing::warn!(
"main_device event with unexpected data length: {}",
device.len()
);
}
}
DmabufFeedbackEvent::FormatTable { .. } => {}
DmabufFeedbackEvent::Done => {}
DmabufFeedbackEvent::TrancheDone => {}
DmabufFeedbackEvent::TrancheTargetDevice { .. } => {}
DmabufFeedbackEvent::TrancheFormats { .. } => {}
DmabufFeedbackEvent::TrancheFlags { .. } => {}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwpLinuxBufferParamsV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwpLinuxBufferParamsV1, ()> for State<S> {
fn event(
state: &mut Self,
proxy: &ZwpLinuxBufferParamsV1,
event: <ZwpLinuxBufferParamsV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
BufferParamsEvent::Created { .. } => {
tracing::debug!("DMA-BUF buffer created");
}
BufferParamsEvent::Failed => {
tracing::error!("DMA-BUF buffer creation failed");
let taken = mem::replace(&mut state.in_flight_surface, InFlightSurface::None);
match taken {
InFlightSurface::CopyQueued { buffer, frame, .. } => {
drop(buffer);
if let EncConstructionStage::Streaming { cap, .. } = &mut state.stage {
cap.on_done_with_frame(frame);
}
}
other => {
state.in_flight_surface = other;
}
}
proxy.destroy();
state.errored = true;
}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrScreencopyFrameV1, ()> for CapWlrScreencopy
// ---------------------------------------------------------------------------
impl Dispatch<ZwlrScreencopyFrameV1, ()> for State<CapWlrScreencopy> {
fn event(
state: &mut Self,
proxy: &ZwlrScreencopyFrameV1,
event: <ZwlrScreencopyFrameV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<CapWlrScreencopy>>,
) {
match event {
// SHM buffer offer — in v3 the compositor enumerates supported buffer
// types (buffer and/or linux_dmabuf) before buffer_done. We only
// support DMA-BUF, so just log and wait for linux_dmabuf / buffer_done.
ScreencopyFrameEvent::Buffer { .. } => {
tracing::debug!("Received SHM Buffer offer — only DMA-BUF capture is supported");
}
ScreencopyFrameEvent::LinuxDmabuf {
format,
width,
height,
} => {
tracing::debug!("Screencopy LinuxDmabuf: format={format}, {width}x{height}");
if !matches!(state.in_flight_surface, InFlightSurface::AllocQueued) {
tracing::warn!("Received LinuxDmabuf while no frame allocation was queued");
return;
}
if matches!(state.stage, EncConstructionStage::EverythingButFmt { .. }) {
state.negotiate_format(format, width, height);
if state.errored {
return;
}
}
if let EncConstructionStage::Streaming { cap, .. } = &mut state.stage {
cap.current_frame = Some(proxy.clone());
}
state.on_frame_allocd((), format, width, height);
}
// v3 terminal event: all buffer offers have been enumerated.
// If still AllocQueued, the compositor never sent linux_dmabuf —
// DMA-BUF screencopy is unsupported, so we must error out.
ScreencopyFrameEvent::BufferDone => {
if matches!(state.in_flight_surface, InFlightSurface::AllocQueued) {
tracing::error!(
"Compositor did not offer DMA-BUF screencopy (only SHM); \
DMA-BUF capture is required"
);
state.in_flight_surface = InFlightSurface::None;
proxy.destroy();
state.errored = true;
}
}
ScreencopyFrameEvent::Ready {
tv_sec_hi,
tv_sec_lo,
tv_nsec,
} => {
let tv_sec = (tv_sec_hi as u64) << 32 | tv_sec_lo as u64;
let tv_usec = tv_nsec / 1000;
tracing::trace!("Screencopy ready: tv_sec={tv_sec}, tv_usec={tv_usec}");
state.on_copy_complete(tv_sec, tv_usec);
}
ScreencopyFrameEvent::Failed => {
tracing::error!("Screencopy frame failed");
state.on_copy_fail();
}
ScreencopyFrameEvent::Damage { .. } => {}
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZxdgOutputManagerV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZxdgOutputManagerV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZxdgOutputManagerV1,
_event: <ZxdgOutputManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrOutputManagerV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwlrOutputManagerV1, ()> for State<S> {
fn event(
state: &mut Self,
_proxy: &ZwlrOutputManagerV1,
event: <ZwlrOutputManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
qhandle: &QueueHandle<State<S>>,
) {
match event {
WlrOutputManagerEvent::Head { head } => {
let _head: ZwlrOutputHeadV1 = head;
tracing::debug!("wlr output head advertised");
}
WlrOutputManagerEvent::Done { .. } => {
if let EncConstructionStage::ProbingOutputs {
wlr_manager_done,
outputs,
..
} = &mut state.stage
{
*wlr_manager_done = true;
let count = outputs.len();
for idx in 0..count {
state.try_finalize_output(idx);
}
}
}
WlrOutputManagerEvent::Finished { .. } => {
tracing::warn!("zwlr_output_manager_v1::Finished received during probing");
}
_ => {}
}
}
event_created_child!(State<S>, ZwlrOutputManagerV1, [
zwlr_output_manager_v1::EVT_HEAD_OPCODE => (ZwlrOutputHeadV1, ()),
]);
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrOutputHeadV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwlrOutputHeadV1, ()> for State<S> {
fn event(
state: &mut Self,
proxy: &ZwlrOutputHeadV1,
event: <ZwlrOutputHeadV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
match event {
WlrHeadEvent::Name { name } => {
if let EncConstructionStage::ProbingOutputs {
wlr_heads,
wlr_head_proxy_to_name,
..
} = &mut state.stage
{
wlr_heads
.entry(name.clone())
.or_insert(WlrHeadInfo { position: None });
wlr_head_proxy_to_name.insert(proxy.id(), name);
}
}
WlrHeadEvent::Position { x, y } => {
if let EncConstructionStage::ProbingOutputs {
wlr_heads,
wlr_head_proxy_to_name,
..
} = &mut state.stage
{
if let Some(name) = wlr_head_proxy_to_name.get(&proxy.id()) {
if let Some(head) = wlr_heads.get_mut(name) {
head.position = Some((x, y));
}
}
}
}
WlrHeadEvent::Finished { .. } => {
tracing::debug!("zwlr_output_head_v1::Finished received");
}
_ => {}
}
}
event_created_child!(State<S>, ZwlrOutputHeadV1, [
zwlr_output_head_v1::EVT_MODE_OPCODE => (ZwlrOutputModeV1, ()),
]);
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrOutputModeV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwlrOutputModeV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwlrOutputModeV1,
_event: <ZwlrOutputModeV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
// ---------------------------------------------------------------------------
// Dispatch<ZwlrScreencopyManagerV1, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<ZwlrScreencopyManagerV1, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &ZwlrScreencopyManagerV1,
_event: <ZwlrScreencopyManagerV1 as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
}
}
// ---------------------------------------------------------------------------
// Dispatch<WlBuffer, ()>
// ---------------------------------------------------------------------------
impl<S: CaptureSource> Dispatch<WlBuffer, ()> for State<S> {
fn event(
_state: &mut Self,
_proxy: &WlBuffer,
event: <WlBuffer as Proxy>::Event,
_data: &(),
_conn: &wayland_client::Connection,
_qhandle: &QueueHandle<State<S>>,
) {
if let wayland_client::protocol::wl_buffer::Event::Release = event {
tracing::trace!("WlBuffer released");
}
}
}
+68 -437
View File
@@ -1,4 +1,8 @@
// 采集门户状态模块 —— 通过 PipeWire/DMA-BUF 进行屏幕采集并编码
// AsRawFd is required by frame.fd.as_raw_fd() in build_drm_descriptor below
// but rustc emits a false "unused_imports" warning because OwnedFd also has
// an inherent as_raw_fd — same quirk as avhw.rs. E0599 if removed → keep it.
#[allow(unused_imports)]
use std::os::fd::AsRawFd;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
@@ -9,13 +13,21 @@ use anyhow::{bail, Result}; // 错误处理工具
use crate::args::Args; // 命令行参数
use crate::avhw::{
self, BitrateCommand, CpuNv12Frame, EncodeOutcome, EncodedH264Frame, ResolutionChange,
SwEncEncode, SwEncImport, SwEncState,
self, BitrateCommand, CpuNv12Frame, ResolutionChange, SwEncEncode, SwEncImport, SwEncState,
}; // 软件编码器状态(VAAPI 导入 + H.264 编码)
use crate::cap_portal::{CapPortal, PwCtrlEvent, PwDmaBufFrame}; // PipeWire 屏幕采集端点
use crate::stats::{FrameTimings, PipelineStats}; // 管道统计(帧计时、每秒快照)
use crate::webrtc::WebRtcState; // WebRTC 信令与媒体传输
mod bitrate;
use bitrate::webrtc_startup_bitrate_bps;
mod threads;
use threads::{
encode_thread_loop, webrtc_thread_loop, EncodeThread, EncodeThreadTiming, WebRtcThreadChannels,
WebRtcThreadConfig, WebrtcThread,
};
/// 门户采集的阶段状态
/// - WaitingForFormat: 等待接收到第一帧 DMA-BUF 以确定视频格式参数
/// - Streaming: 已完成初始化,正在持续编码流
@@ -24,24 +36,6 @@ enum PortalStage {
Streaming,
}
struct EncodeThreadTiming {
sws_us: u64,
encode_us: u64,
output_bytes: usize,
}
struct EncodeThread {
handle: Option<std::thread::JoinHandle<()>>,
input_tx: crossbeam_channel::Sender<CpuNv12Frame>,
timing_rx: crossbeam_channel::Receiver<EncodeThreadTiming>,
duplicate_count: std::sync::Arc<std::sync::atomic::AtomicU64>,
}
struct WebrtcThread {
handle: Option<std::thread::JoinHandle<()>>,
sent_gap_rx: crossbeam_channel::Receiver<(f64, Option<f64>)>,
}
/// 门户模式的主状态机
///
/// 负责管理从 PipeWire 采集屏幕帧、通过 VAAPI 硬件编码的完整生命周期。
@@ -249,9 +243,7 @@ impl StatePortal {
bitrate_rx,
encoder_resolution_rx,
)?;
let duplicate_count = std::sync::Arc::new(
std::sync::atomic::AtomicU64::new(0),
);
let duplicate_count = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0));
let duplicate_count_for_thread = duplicate_count.clone();
let handle = std::thread::Builder::new()
.name("wl-webrtc-encode".into())
@@ -288,15 +280,19 @@ impl StatePortal {
.spawn(move || {
webrtc_thread_loop(
wrtc,
webrtc_rx,
fps,
enc_width,
enc_height,
max_bitrate,
WebRtcThreadConfig {
fps,
enc_width,
enc_height,
max_bitrate,
},
WebRtcThreadChannels {
webrtc_rx,
sent_gap_tx,
bitrate_tx,
resolution_tx,
},
paused,
sent_gap_tx,
bitrate_tx,
resolution_tx,
)
})?;
self.webrtc_thread = Some(WebrtcThread {
@@ -338,8 +334,17 @@ impl StatePortal {
// 每秒输出一次结构化管道统计(仅 --stats 启用时记录日志)
if self.args.stats && self.stats.should_snapshot() {
self.stats.set_pipewire_dropped(0, 0);
self.stats.set_queue_depths(0, 0);
// Wire PipeWire drop counter (delta-tracked via pw_dropped_prev) and
// capture channel depth. Oracle audit 2026-06-28: previously hardcoded
// (0, 0), which silently zeroed two real diagnostic fields.
let total_dropped = self.cap.dropped_count();
self.stats
.set_pipewire_dropped(total_dropped, self.pw_dropped_prev);
self.pw_dropped_prev = total_dropped;
// capture queue depth is real; encoded side has no exposed depth — the
// encoder thread publishes timings only, not a frame queue length.
self.stats
.set_queue_depths(self.cap.capture_queue_depth(), 0);
if let Some(ref enc_thread) = self.enc_thread {
while let Ok(timing) = enc_thread.timing_rx.try_recv() {
self.stats.record_encode_thread(
@@ -478,55 +483,45 @@ impl StatePortal {
if let Some(enc) = self.enc.as_mut() {
// 将 DMA-BUF 帧零拷贝导入 VAAPI 硬件帧池
let mut vaapi_frame = unsafe {
avhw::import_dma_buf_to_vaapi(
enc.frames_rgb().as_ptr(),
frame.fd.as_raw_fd(),
frame.width,
frame.height,
frame.format,
frame.modifier,
frame.stride,
frame.offset,
)
}?;
// SAFETY: delegates to avhw::import_dma_buf_to_vaapi (itself an unsafe fn);
// frames_rgb pointer is a valid AVBufferRef owned by enc, and `frame` is the
// PipeWire-formatted PwDmaBufFrame whose metadata the function reads directly.
// See that function's own SAFETY contract.
let mut vaapi_frame =
unsafe { avhw::import_dma_buf_to_vaapi(enc.frames_rgb().as_ptr(), &frame) }?;
let import_us = t_import_start.elapsed().as_micros() as u64;
let t_encode_start = Instant::now();
// 设置帧的显示时间戳(PTS),基于已编码帧序号
// SAFETY: vaapi_frame is the freshly-imported valid AVFrame returned by
// import_dma_buf_to_vaapi above; pts is a plain i64 field on AVFrame.
unsafe {
(*vaapi_frame.as_mut_ptr()).pts = pts;
}
// 送入编码器完成:缩放 → 回读 → 格式转换 → H.264 编码
enc.encode_frame(&vaapi_frame)?;
let stages = enc.encode_frame(&vaapi_frame)?;
let total_us = t_import_start.elapsed().as_micros() as u64;
let encode_us = t_encode_start.elapsed().as_micros() as u64;
let encode_us = stages.encode_us;
self.frames_encoded += 1;
// 记录帧计时到管道统计(import + encode 内部各阶段暂不可分离,用 total 覆盖
// 记录帧计时到管道统计(scale 来自 filter graphtransfer 在 HW 路径恒为 0
let timings = FrameTimings {
import_us,
scale_us: stages.scale_us,
transfer_us: stages.transfer_us,
encode_us,
total_us,
..Default::default()
};
self.stats.record_encode(&timings);
} else if let Some(import) = self.enc_import.as_mut() {
let mut vaapi_frame = unsafe {
avhw::import_dma_buf_to_vaapi(
import.frames_rgb().as_ptr(),
frame.fd.as_raw_fd(),
frame.width,
frame.height,
frame.format,
frame.modifier,
frame.stride,
frame.offset,
)
}?;
// SAFETY: same contract as the enc branch above — frames_rgb owned by
// import, `frame` carries the PipeWire DMA-BUF metadata.
let mut vaapi_frame =
unsafe { avhw::import_dma_buf_to_vaapi(import.frames_rgb().as_ptr(), &frame) }?;
// SAFETY: vaapi_frame is the valid AVFrame returned above; pts is plain i64.
unsafe {
(*vaapi_frame.as_mut_ptr()).pts = pts;
}
@@ -652,253 +647,6 @@ impl StatePortal {
}
}
fn encode_thread_loop(
mut encode: SwEncEncode,
input_rx: crossbeam_channel::Receiver<CpuNv12Frame>,
timing_tx: crossbeam_channel::Sender<EncodeThreadTiming>,
duplicate_count: std::sync::Arc<std::sync::atomic::AtomicU64>,
) {
loop {
match input_rx.recv() {
Ok(frame) => {
match encode.encode_cpu_frame(&frame) {
Ok(EncodeOutcome::Encoded) => {
let t = encode.take_timing();
let _ = timing_tx.try_send(EncodeThreadTiming {
sws_us: t.sws_us,
encode_us: t.encode_us,
output_bytes: t.output_bytes,
});
}
Ok(EncodeOutcome::SkippedDuplicate) => {
duplicate_count.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
}
Ok(_) => {
// SkippedPaused / SkippedDisconnected — no counter needed
}
Err(e) => {
tracing::error!("Encode thread error: {e}");
break;
}
}
}
Err(_) => {
tracing::info!("Encode thread input closed, flushing encoder");
if let Err(e) = encode.flush() {
tracing::error!("Encode thread flush error: {e}");
}
break;
}
}
}
tracing::info!("Encode thread exiting");
}
fn webrtc_thread_loop(
mut wrtc: WebRtcState,
webrtc_rx: crossbeam_channel::Receiver<EncodedH264Frame>,
fps: u32,
enc_width: u32,
enc_height: u32,
max_bitrate: u64,
paused: Arc<AtomicBool>,
sent_gap_tx: crossbeam_channel::Sender<(f64, Option<f64>)>,
bitrate_tx: crossbeam_channel::Sender<BitrateCommand>,
resolution_tx: crossbeam_channel::Sender<BitrateCommand>,
) {
let mut frames_sent: u64 = 0;
let mut last_send: Option<std::time::Instant> = None;
let mut last_sent_bitrate: Option<u64> = None;
let initial_tier = (enc_width, enc_height);
let mut current_tier = initial_tier;
let mut upscale_counter = 0u32;
let mut last_resolution_eval = Instant::now();
let timeout = Duration::from_millis(1);
loop {
if let Err(e) = wrtc.handle_signaling() {
tracing::error!("WebRTC signaling error: {e}");
break;
}
if let Err(e) = wrtc.poll_and_feed() {
tracing::error!("WebRTC poll error: {e}");
break;
}
if wrtc.take_force_keyframe() {
let _ = bitrate_tx.try_send(BitrateCommand::ForceKeyframe);
}
let connected = wrtc.is_connected();
let was_paused = paused.load(Ordering::Relaxed);
let now_paused = !connected;
if was_paused && !now_paused {
tracing::info!("WebRTC client connected, resuming encoding");
} else if !was_paused && now_paused {
tracing::warn!("WebRTC client disconnected, pausing encoding");
}
paused.store(now_paused, Ordering::Relaxed);
if let Some(bwe) = wrtc.get_bwe_estimate() {
// #23: Cap BWE to prevent runaway bitrate escalation. Without this, BWE
// estimates can rise to 10+ Mbps, causing IDR bursts and PLI storms.
let effective_bwe = bwe.min(max_bitrate);
if effective_bwe != bwe {
tracing::debug!(
bwe,
effective_bwe,
max_bitrate,
"BWE exceeds --max-bitrate cap, clamping"
);
}
let bwe = effective_bwe;
let should_send = match last_sent_bitrate {
None => true,
Some(last) => {
let diff = if bwe > last { bwe - last } else { last - bwe };
diff * 10 > last
}
};
if should_send {
let _ = bitrate_tx.try_send(BitrateCommand::UpdateBitrate { target_bps: bwe });
last_sent_bitrate = Some(bwe);
}
if last_resolution_eval.elapsed() >= Duration::from_secs(1) {
last_resolution_eval = Instant::now();
let selected = select_resolution(current_tier.0, current_tier.1, bwe, fps);
if selected != current_tier {
current_tier = selected;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
} else if let Some(next_tier) = next_upscale_tier(current_tier, initial_tier) {
let needed = resolution_bitrate_bps(next_tier.0, next_tier.1, fps);
if bwe > needed.saturating_mul(120) / 100 {
upscale_counter = upscale_counter.saturating_add(1);
if upscale_counter >= 10 {
current_tier = next_tier;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
}
} else {
upscale_counter = 0;
}
} else {
upscale_counter = 0;
}
}
}
if connected {
while let Ok(enc_frame) = webrtc_rx.try_recv() {
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks) {
tracing::debug!("WebRTC write frame error: {e}");
}
frames_sent = frames_sent.saturating_add(1);
let gap_ms = last_send
.map(|l| l.elapsed().as_secs_f64() * 1000.0)
.unwrap_or(0.0);
// Compute capture-to-send age on the sending thread so the
// frame_age stat stays accurate when batch-drained later.
let age_ms =
Some(enc_frame.capture_time.elapsed().as_secs_f64() * 1000.0);
last_send = Some(std::time::Instant::now());
let _ = sent_gap_tx.try_send((gap_ms, age_ms));
}
} else {
while webrtc_rx.try_recv().is_ok() {}
}
match webrtc_rx.recv_timeout(timeout) {
Ok(enc_frame) => {
if wrtc.is_connected() {
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks)
{
tracing::debug!("WebRTC write frame error: {e}");
}
frames_sent = frames_sent.saturating_add(1);
let gap_ms = last_send
.map(|l| l.elapsed().as_secs_f64() * 1000.0)
.unwrap_or(0.0);
let age_ms =
Some(enc_frame.capture_time.elapsed().as_secs_f64() * 1000.0);
last_send = Some(std::time::Instant::now());
let _ = sent_gap_tx.try_send((gap_ms, age_ms));
}
}
Err(crossbeam_channel::RecvTimeoutError::Timeout) => {}
Err(crossbeam_channel::RecvTimeoutError::Disconnected) => {
tracing::info!("WebRTC channel disconnected, exiting thread");
return;
}
}
}
tracing::info!("WebRTC thread exiting");
}
const RESOLUTION_TIERS: &[(u32, u32)] = &[(2560, 1440), (1920, 1080), (1280, 720)];
fn resolution_bitrate_bps(width: u32, height: u32, fps: u32) -> u64 {
5 * u64::from(width) * u64::from(height) * u64::from(fps) / 100
}
/// Conservative startup bitrate for WebRTC mode, tier-based by total pixel count.
/// BWE estimate arrives within milliseconds of client connect and overrides this;
/// the startup value only affects the first IDR. See issue #21.
fn webrtc_startup_bitrate_bps(width: u32, height: u32) -> u64 {
let pixels = u64::from(width) * u64::from(height);
if pixels <= 1_000_000 {
1_000_000
} else if pixels <= 2_500_000 {
2_000_000
} else if pixels <= 4_500_000 {
4_000_000
} else {
8_000_000
}
}
/// Select resolution tier based on BWE estimate.
/// Returns (width, height) for the selected tier.
fn select_resolution(current_w: u32, current_h: u32, bwe_bps: u64, fps: u32) -> (u32, u32) {
let current = (current_w, current_h);
let current_bitrate = resolution_bitrate_bps(current_w, current_h, fps);
if bwe_bps >= current_bitrate.saturating_mul(60) / 100 {
return current;
}
let current_index = RESOLUTION_TIERS
.iter()
.position(|&tier| tier == current)
.unwrap_or_else(|| {
RESOLUTION_TIERS
.iter()
.position(|&(w, h)| w <= current_w && h <= current_h)
.unwrap_or(RESOLUTION_TIERS.len() - 1)
});
let next_index = (current_index + 1).min(RESOLUTION_TIERS.len() - 1);
RESOLUTION_TIERS[next_index]
}
fn next_upscale_tier(current: (u32, u32), ceiling: (u32, u32)) -> Option<(u32, u32)> {
let current_index = RESOLUTION_TIERS.iter().position(|&tier| tier == current)?;
if current_index == 0 {
return None;
}
let next = RESOLUTION_TIERS[current_index - 1];
(next.0 <= ceiling.0 && next.1 <= ceiling.1).then_some(next)
}
impl Drop for StatePortal {
// 析构时自动调用 shutdown,确保编码器被刷新、资源被释放
fn drop(&mut self) {
@@ -946,7 +694,12 @@ fn resolve_drm_device(args: &Args) -> Result<Option<PathBuf>> {
/// 用于验证 DMA-BUF 元数据映射的正确性。
#[cfg(test)]
fn build_drm_descriptor(frame: &PwDmaBufFrame) -> ffmpeg_next::ffi::AVDRMFrameDescriptor {
let mut desc: ffmpeg_next::ffi::AVDRMFrameDescriptor = unsafe { std::mem::zeroed() };
let mut desc: ffmpeg_next::ffi::AVDRMFrameDescriptor = {
// SAFETY: AVDRMFrameDescriptor is a POD struct from FFmpeg's C API with no
// pointers orDrop fields; all-zero is a valid initial state. Every field is
// explicitly overwritten in the lines below before the descriptor is used.
unsafe { std::mem::zeroed() }
};
desc.nb_objects = 1; // 单个 DMA-BUF 对象
desc.objects[0].fd = frame.fd.as_raw_fd(); // DMA-BUF 文件描述符
desc.objects[0].size = 0; // 大小设为 0(内核自动确定)
@@ -969,6 +722,9 @@ mod tests {
fn make_test_frame() -> PwDmaBufFrame {
// Create a dummy fd from stderr (always valid fd 2)
// 使用 stderr(fd 2)的副本作为虚拟文件描述符
// SAFETY: stderr (fd 2) is always-open in any process; libc::dup(2) returns
// a fresh fd we solely own. OwnedFd::from_raw_fd takes ownership and closes
// it on Drop. Test-only; the fd is never actually memory-mapped.
let fd = unsafe { OwnedFd::from_raw_fd(libc::dup(2)) };
PwDmaBufFrame {
fd,
@@ -1046,53 +802,13 @@ mod tests {
assert_eq!(result, None);
}
#[test]
fn webrtc_startup_bitrate_tiers_by_pixel_count() {
assert_eq!(webrtc_startup_bitrate_bps(1280, 720), 1_000_000);
assert_eq!(webrtc_startup_bitrate_bps(1920, 1080), 2_000_000);
assert_eq!(webrtc_startup_bitrate_bps(2560, 1440), 4_000_000);
assert_eq!(webrtc_startup_bitrate_bps(3840, 2160), 8_000_000);
}
#[test]
fn select_resolution_downscales_one_tier_below_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 59 / 100, fps),
(1280, 720)
);
}
#[test]
fn select_resolution_keeps_tier_at_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 60 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_never_goes_below_720p() {
assert_eq!(select_resolution(1280, 720, 1, 30), (1280, 720));
}
#[test]
fn next_upscale_tier_respects_initial_ceiling() {
assert_eq!(
next_upscale_tier((1280, 720), (1920, 1080)),
Some((1920, 1080))
);
assert_eq!(next_upscale_tier((1920, 1080), (1920, 1080)), None);
}
/// 测试:使用自定义偏移量和 stride 构建 DRM 描述符
#[test]
fn build_drm_descriptor_custom_offset_and_stride() {
// SAFETY: same as make_test_frame — dup of stderr (fd 2), test-only.
let test_fd = unsafe { OwnedFd::from_raw_fd(libc::dup(2)) };
let frame = PwDmaBufFrame {
fd: unsafe { OwnedFd::from_raw_fd(libc::dup(2)) },
fd: test_fd,
offset: 4096, // 4KB 对齐偏移
stride: 3840 * 4, // 4K 宽度 × 4 字节
modifier: 0x0100000000000001, // AMD modifiers
@@ -1110,89 +826,4 @@ mod tests {
}
// ── issue #8 regression ──
#[test]
fn try_send_full_channel_returns_full_not_block() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
tx.send(vec![1]).unwrap();
tx.send(vec![2]).unwrap();
assert!(matches!(
tx.try_send(vec![3]),
Err(crossbeam_channel::TrySendError::Full(_))
));
assert_eq!(rx.len(), 2);
}
#[test]
fn try_send_after_rx_dropped_returns_disconnected() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
drop(rx);
assert!(matches!(
tx.try_send(vec![1]),
Err(crossbeam_channel::TrySendError::Disconnected(_))
));
}
// given: full bounded channel
// when: rx is dropped, then try_send
// expect: Disconnected, not blocking
#[test]
fn shutdown_rx_drop_prevents_deadlock_on_full_channel() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
tx.send(vec![1]).unwrap();
tx.send(vec![2]).unwrap();
drop(rx);
assert!(matches!(
tx.try_send(vec![3]),
Err(crossbeam_channel::TrySendError::Disconnected(_))
));
}
// ── Task 7: Additional resolution tier edge cases ──
#[test]
fn select_resolution_keeps_720p_when_bwe_sufficient() {
let fps = 30;
let bitrate_720 = resolution_bitrate_bps(1280, 720, fps);
assert_eq!(
select_resolution(1280, 720, bitrate_720, fps),
(1280, 720)
);
}
#[test]
fn select_resolution_downscales_1440p_to_1080p() {
let fps = 30;
let bitrate_1440 = resolution_bitrate_bps(2560, 1440, fps);
assert_eq!(
select_resolution(2560, 1440, bitrate_1440 * 59 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_1080p_to_720p_at_very_low_bwe() {
let fps = 30;
let bitrate_1080 = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, bitrate_1080 / 10, fps),
(1280, 720)
);
}
#[test]
fn next_upscale_tier_from_720p_to_1080p() {
assert_eq!(
next_upscale_tier((1280, 720), (2560, 1440)),
Some((1920, 1080))
);
}
#[test]
fn next_upscale_tier_returns_none_at_highest() {
assert_eq!(next_upscale_tier((2560, 1440), (2560, 1440)), None);
}
}
+144
View File
@@ -0,0 +1,144 @@
pub(super) const RESOLUTION_TIERS: &[(u32, u32)] = &[(2560, 1440), (1920, 1080), (1280, 720)];
pub(super) fn resolution_bitrate_bps(width: u32, height: u32, fps: u32) -> u64 {
5 * u64::from(width) * u64::from(height) * u64::from(fps) / 100
}
/// Conservative startup bitrate for WebRTC mode, tier-based by total pixel count.
/// BWE estimate arrives within milliseconds of client connect and overrides this;
/// the startup value only affects the first IDR. See issue #21.
pub(super) fn webrtc_startup_bitrate_bps(width: u32, height: u32) -> u64 {
let pixels = u64::from(width) * u64::from(height);
if pixels <= 1_000_000 {
1_000_000
} else if pixels <= 2_500_000 {
2_000_000
} else if pixels <= 4_500_000 {
4_000_000
} else {
8_000_000
}
}
/// Select resolution tier based on BWE estimate.
/// Returns (width, height) for the selected tier.
pub(super) fn select_resolution(
current_w: u32,
current_h: u32,
bwe_bps: u64,
fps: u32,
) -> (u32, u32) {
let current = (current_w, current_h);
let current_bitrate = resolution_bitrate_bps(current_w, current_h, fps);
if bwe_bps >= current_bitrate.saturating_mul(60) / 100 {
return current;
}
let current_index = RESOLUTION_TIERS
.iter()
.position(|&tier| tier == current)
.unwrap_or_else(|| {
RESOLUTION_TIERS
.iter()
.position(|&(w, h)| w <= current_w && h <= current_h)
.unwrap_or(RESOLUTION_TIERS.len() - 1)
});
let next_index = (current_index + 1).min(RESOLUTION_TIERS.len() - 1);
RESOLUTION_TIERS[next_index]
}
pub(super) fn next_upscale_tier(current: (u32, u32), ceiling: (u32, u32)) -> Option<(u32, u32)> {
let current_index = RESOLUTION_TIERS.iter().position(|&tier| tier == current)?;
if current_index == 0 {
return None;
}
let next = RESOLUTION_TIERS[current_index - 1];
(next.0 <= ceiling.0 && next.1 <= ceiling.1).then_some(next)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn webrtc_startup_bitrate_tiers_by_pixel_count() {
assert_eq!(webrtc_startup_bitrate_bps(1280, 720), 1_000_000);
assert_eq!(webrtc_startup_bitrate_bps(1920, 1080), 2_000_000);
assert_eq!(webrtc_startup_bitrate_bps(2560, 1440), 4_000_000);
assert_eq!(webrtc_startup_bitrate_bps(3840, 2160), 8_000_000);
}
#[test]
fn select_resolution_downscales_one_tier_below_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 59 / 100, fps),
(1280, 720)
);
}
#[test]
fn select_resolution_keeps_tier_at_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 60 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_never_goes_below_720p() {
assert_eq!(select_resolution(1280, 720, 1, 30), (1280, 720));
}
#[test]
fn next_upscale_tier_respects_initial_ceiling() {
assert_eq!(
next_upscale_tier((1280, 720), (1920, 1080)),
Some((1920, 1080))
);
assert_eq!(next_upscale_tier((1920, 1080), (1920, 1080)), None);
}
#[test]
fn select_resolution_keeps_720p_when_bwe_sufficient() {
let fps = 30;
let bitrate_720 = resolution_bitrate_bps(1280, 720, fps);
assert_eq!(select_resolution(1280, 720, bitrate_720, fps), (1280, 720));
}
#[test]
fn select_resolution_downscales_1440p_to_1080p() {
let fps = 30;
let bitrate_1440 = resolution_bitrate_bps(2560, 1440, fps);
assert_eq!(
select_resolution(2560, 1440, bitrate_1440 * 59 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_1080p_to_720p_at_very_low_bwe() {
let fps = 30;
let bitrate_1080 = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, bitrate_1080 / 10, fps),
(1280, 720)
);
}
#[test]
fn next_upscale_tier_from_720p_to_1080p() {
assert_eq!(
next_upscale_tier((1280, 720), (2560, 1440)),
Some((1920, 1080))
);
}
#[test]
fn next_upscale_tier_returns_none_at_highest() {
assert_eq!(next_upscale_tier((2560, 1440), (2560, 1440)), None);
}
}
+287
View File
@@ -0,0 +1,287 @@
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
use crate::avhw::{BitrateCommand, CpuNv12Frame, EncodeOutcome, EncodedH264Frame, SwEncEncode};
use crate::webrtc::WebRtcState;
use super::bitrate::{next_upscale_tier, resolution_bitrate_bps, select_resolution};
pub(super) struct EncodeThreadTiming {
pub(super) sws_us: u64,
pub(super) encode_us: u64,
pub(super) output_bytes: usize,
}
pub(super) struct EncodeThread {
pub(super) handle: Option<std::thread::JoinHandle<()>>,
pub(super) input_tx: crossbeam_channel::Sender<CpuNv12Frame>,
pub(super) timing_rx: crossbeam_channel::Receiver<EncodeThreadTiming>,
pub(super) duplicate_count: std::sync::Arc<std::sync::atomic::AtomicU64>,
}
pub(super) struct WebrtcThread {
pub(super) handle: Option<std::thread::JoinHandle<()>>,
pub(super) sent_gap_rx: crossbeam_channel::Receiver<(f64, Option<f64>)>,
}
/// Static configuration handed to the WebRTC sender thread. Immutable for the
/// thread's lifetime; a resolution tier change rebuilds the whole pipeline
/// (and spawns a new thread) rather than mutating this.
pub(super) struct WebRtcThreadConfig {
pub(super) fps: u32,
pub(super) enc_width: u32,
pub(super) enc_height: u32,
pub(super) max_bitrate: u64,
}
/// Channel endpoints owned exclusively by the WebRTC sender thread after spawn.
/// The reverse endpoints stay with StatePortal (or the encode thread) for
/// inbound/outbound traffic.
pub(super) struct WebRtcThreadChannels {
pub(super) webrtc_rx: crossbeam_channel::Receiver<EncodedH264Frame>,
pub(super) sent_gap_tx: crossbeam_channel::Sender<(f64, Option<f64>)>,
pub(super) bitrate_tx: crossbeam_channel::Sender<BitrateCommand>,
pub(super) resolution_tx: crossbeam_channel::Sender<BitrateCommand>,
}
pub(super) fn encode_thread_loop(
mut encode: SwEncEncode,
input_rx: crossbeam_channel::Receiver<CpuNv12Frame>,
timing_tx: crossbeam_channel::Sender<EncodeThreadTiming>,
duplicate_count: std::sync::Arc<std::sync::atomic::AtomicU64>,
) {
loop {
match input_rx.recv() {
Ok(frame) => {
match encode.encode_cpu_frame(&frame) {
Ok(EncodeOutcome::Encoded) => {
let t = encode.take_timing();
let _ = timing_tx.try_send(EncodeThreadTiming {
sws_us: t.sws_us,
encode_us: t.encode_us,
output_bytes: t.output_bytes,
});
}
Ok(EncodeOutcome::SkippedDuplicate) => {
duplicate_count.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
}
Ok(_) => {
// SkippedPaused / SkippedDisconnected — no counter needed
}
Err(e) => {
tracing::error!("Encode thread error: {e}");
break;
}
}
}
Err(_) => {
tracing::info!("Encode thread input closed, flushing encoder");
if let Err(e) = encode.flush() {
tracing::error!("Encode thread flush error: {e}");
}
break;
}
}
}
tracing::info!("Encode thread exiting");
}
pub(super) fn webrtc_thread_loop(
mut wrtc: WebRtcState,
config: WebRtcThreadConfig,
channels: WebRtcThreadChannels,
paused: Arc<AtomicBool>,
) {
let WebRtcThreadConfig {
fps,
enc_width,
enc_height,
max_bitrate,
} = config;
let WebRtcThreadChannels {
webrtc_rx,
sent_gap_tx,
bitrate_tx,
resolution_tx,
} = channels;
let mut frames_sent: u64 = 0;
let mut last_send: Option<std::time::Instant> = None;
let mut last_sent_bitrate: Option<u64> = None;
let initial_tier = (enc_width, enc_height);
let mut current_tier = initial_tier;
let mut upscale_counter = 0u32;
let mut last_resolution_eval = Instant::now();
let timeout = Duration::from_millis(1);
loop {
if let Err(e) = wrtc.handle_signaling() {
tracing::error!("WebRTC signaling error: {e}");
break;
}
if let Err(e) = wrtc.poll_and_feed() {
tracing::error!("WebRTC poll error: {e}");
break;
}
if wrtc.take_force_keyframe() {
let _ = bitrate_tx.try_send(BitrateCommand::ForceKeyframe);
}
let connected = wrtc.is_connected();
let was_paused = paused.load(Ordering::Relaxed);
let now_paused = !connected;
if was_paused && !now_paused {
tracing::info!("WebRTC client connected, resuming encoding");
} else if !was_paused && now_paused {
tracing::warn!("WebRTC client disconnected, pausing encoding");
}
paused.store(now_paused, Ordering::Relaxed);
if let Some(bwe) = wrtc.get_bwe_estimate() {
// #23: Cap BWE to prevent runaway bitrate escalation. Without this, BWE
// estimates can rise to 10+ Mbps, causing IDR bursts and PLI storms.
let effective_bwe = bwe.min(max_bitrate);
if effective_bwe != bwe {
tracing::debug!(
bwe,
effective_bwe,
max_bitrate,
"BWE exceeds --max-bitrate cap, clamping"
);
}
let bwe = effective_bwe;
let should_send = match last_sent_bitrate {
None => true,
Some(last) => {
let diff = bwe.abs_diff(last);
diff * 10 > last
}
};
if should_send {
let _ = bitrate_tx.try_send(BitrateCommand::UpdateBitrate { target_bps: bwe });
last_sent_bitrate = Some(bwe);
}
if last_resolution_eval.elapsed() >= Duration::from_secs(1) {
last_resolution_eval = Instant::now();
let selected = select_resolution(current_tier.0, current_tier.1, bwe, fps);
if selected != current_tier {
current_tier = selected;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
} else if let Some(next_tier) = next_upscale_tier(current_tier, initial_tier) {
let needed = resolution_bitrate_bps(next_tier.0, next_tier.1, fps);
if bwe > needed.saturating_mul(120) / 100 {
upscale_counter = upscale_counter.saturating_add(1);
if upscale_counter >= 10 {
current_tier = next_tier;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
}
} else {
upscale_counter = 0;
}
} else {
upscale_counter = 0;
}
}
}
if connected {
while let Ok(enc_frame) = webrtc_rx.try_recv() {
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks) {
tracing::debug!("WebRTC write frame error: {e}");
}
frames_sent = frames_sent.saturating_add(1);
let gap_ms = last_send
.map(|l| l.elapsed().as_secs_f64() * 1000.0)
.unwrap_or(0.0);
// Compute capture-to-send age on the sending thread so the
// frame_age stat stays accurate when batch-drained later.
let age_ms = Some(enc_frame.capture_time.elapsed().as_secs_f64() * 1000.0);
last_send = Some(std::time::Instant::now());
let _ = sent_gap_tx.try_send((gap_ms, age_ms));
}
} else {
while webrtc_rx.try_recv().is_ok() {}
}
match webrtc_rx.recv_timeout(timeout) {
Ok(enc_frame) => {
if wrtc.is_connected() {
if let Err(e) = wrtc.write_h264_frame(&enc_frame.data, enc_frame.pts_ticks) {
tracing::debug!("WebRTC write frame error: {e}");
}
frames_sent = frames_sent.saturating_add(1);
let gap_ms = last_send
.map(|l| l.elapsed().as_secs_f64() * 1000.0)
.unwrap_or(0.0);
let age_ms = Some(enc_frame.capture_time.elapsed().as_secs_f64() * 1000.0);
last_send = Some(std::time::Instant::now());
let _ = sent_gap_tx.try_send((gap_ms, age_ms));
}
}
Err(crossbeam_channel::RecvTimeoutError::Timeout) => {}
Err(crossbeam_channel::RecvTimeoutError::Disconnected) => {
tracing::info!("WebRTC channel disconnected, exiting thread");
return;
}
}
}
tracing::info!("WebRTC thread exiting");
}
#[cfg(test)]
mod tests {
#[test]
fn try_send_full_channel_returns_full_not_block() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
tx.send(vec![1]).unwrap();
tx.send(vec![2]).unwrap();
assert!(matches!(
tx.try_send(vec![3]),
Err(crossbeam_channel::TrySendError::Full(_))
));
assert_eq!(rx.len(), 2);
}
#[test]
fn try_send_after_rx_dropped_returns_disconnected() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
drop(rx);
assert!(matches!(
tx.try_send(vec![1]),
Err(crossbeam_channel::TrySendError::Disconnected(_))
));
}
// given: full bounded channel
// when: rx is dropped, then try_send
// expect: Disconnected, not blocking
#[test]
fn shutdown_rx_drop_prevents_deadlock_on_full_channel() {
let (tx, rx) = crossbeam_channel::bounded::<Vec<u8>>(2);
tx.send(vec![1]).unwrap();
tx.send(vec![2]).unwrap();
drop(rx);
assert!(matches!(
tx.try_send(vec![3]),
Err(crossbeam_channel::TrySendError::Disconnected(_))
));
}
}
+49 -23
View File
@@ -38,7 +38,6 @@ pub struct PipelineStats {
encoded_frames: u64,
sent_frames: u64,
pipewire_dropped: u64,
over_budget_count: u64,
/// Count of frames dropped by encode thread due to Y-plane hash dedup
/// (EncodeOutcome::SkippedDuplicate). Read from atomic counter set by
/// encode thread, computed as delta since previous snapshot.
@@ -73,6 +72,12 @@ pub struct PipelineStats {
window_start: Instant,
}
impl Default for PipelineStats {
fn default() -> Self {
Self::new()
}
}
impl PipelineStats {
pub fn new() -> Self {
Self {
@@ -80,7 +85,6 @@ impl PipelineStats {
encoded_frames: 0,
sent_frames: 0,
pipewire_dropped: 0,
over_budget_count: 0,
duplicate_frames_skipped: 0,
prev_duplicate_frames_skipped: 0,
capture_queue_depth: 0,
@@ -197,7 +201,8 @@ impl PipelineStats {
/// Update duplicate frames skipped counter (absolute value from atomic).
/// Computes delta from previous value, like set_pipewire_dropped.
pub fn set_duplicate_frames_skipped(&mut self, total_skipped: u64) {
self.duplicate_frames_skipped = total_skipped.saturating_sub(self.prev_duplicate_frames_skipped);
self.duplicate_frames_skipped =
total_skipped.saturating_sub(self.prev_duplicate_frames_skipped);
self.prev_duplicate_frames_skipped = total_skipped;
}
@@ -207,11 +212,6 @@ impl PipelineStats {
self.encoded_queue_depth = encoded;
}
/// Record that a frame exceeded its time budget.
pub fn record_over_budget(&mut self) {
self.over_budget_count += 1;
}
/// Returns true if at least 1 second has elapsed since the last snapshot
/// (or since creation). If true, call `snapshot_and_reset` to get the stats.
pub fn should_snapshot(&self) -> bool {
@@ -230,7 +230,6 @@ impl PipelineStats {
encoded_frames: self.encoded_frames,
sent_frames: self.sent_frames,
pipewire_dropped: self.pipewire_dropped,
over_budget_count: self.over_budget_count,
duplicate_frames_skipped: self.duplicate_frames_skipped,
capture_queue_depth: self.capture_queue_depth,
encoded_queue_depth: self.encoded_queue_depth,
@@ -269,7 +268,6 @@ impl PipelineStats {
self.encoded_frames = 0;
self.sent_frames = 0;
self.pipewire_dropped = 0;
self.over_budget_count = 0;
self.duplicate_frames_skipped = 0;
self.capture_queue_depth = 0;
self.encoded_queue_depth = 0;
@@ -304,7 +302,6 @@ pub struct StatsSnapshot {
pub encoded_frames: u64,
pub sent_frames: u64,
pub pipewire_dropped: u64,
pub over_budget_count: u64,
pub duplicate_frames_skipped: u64,
// Queue depths
pub capture_queue_depth: usize,
@@ -346,43 +343,72 @@ pub struct StatsSnapshot {
impl std::fmt::Display for StatsSnapshot {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
// Layout note: each line answers one operational question.
// - line 1: throughput (fps + frame counts + window length)
// - line 2: drops (PipeWire backlog, encoder over-budget, frame-hash dedup)
// - line 3: queue back-pressure (capture + encoded)
// - line 4-6: gap timing (avg + p95 + max) — answers "is cadence stable?"
// - line 7: capture-to-send age (avg + p95 + max) — answers "how stale?"
// - line 8: per-stage encode timing (avg + p95) — answers "where is latency?"
// - line 9: output bandwidth (bytes/sec + per-frame p95/max)
//
// The avg counterparts were computed but never displayed before Oracle
// audit 2026-06-28; they pair with the existing p95/max to surface both
// central tendency and tail behaviour in the same glance.
write!(
f,
"capture_fps={:.1} encoded_fps={:.1} sent_fps={:.1} \
pw_dropped={} over_budget={} duplicate_frames_skipped={} \
cap_q={} enc_q={} \
cap_gap_p95={:.1}ms cap_gap_max={:.1}ms \
enc_gap_p95={:.1}ms enc_gap_max={:.1}ms \
sent_gap_p95={:.1}ms sent_gap_max={:.1}ms \
frame_age_p95={:.1}ms frame_age_max={:.1}ms \
send_wait_p95={:.1}ms \
import_p95={:.1}ms scale_p95={:.1}ms transfer_p95={:.1}ms \
sws_p95={:.1}ms encode_p95={:.1}ms total_p95={:.1}ms \
output_bps={:.0} frame_bytes_max={}",
"elapsed={:.1}s capture_fps={:.1} encoded_fps={:.1} sent_fps={:.1} \
capture_frames={} encoded_frames={} sent_frames={} \
pw_dropped={} duplicate_frames_skipped={} \
cap_q={} enc_q={} \
cap_gap_avg={:.1}ms cap_gap_p95={:.1}ms cap_gap_max={:.1}ms \
enc_gap_avg={:.1}ms enc_gap_p95={:.1}ms enc_gap_max={:.1}ms \
sent_gap_avg={:.1}ms sent_gap_p95={:.1}ms sent_gap_max={:.1}ms \
frame_age_avg={:.1}ms frame_age_p95={:.1}ms frame_age_max={:.1}ms \
send_wait_p95={:.1}ms \
import_avg={:.1}ms import_p95={:.1}ms \
scale_avg={:.1}ms scale_p95={:.1}ms transfer_avg={:.1}ms transfer_p95={:.1}ms \
sws_avg={:.1}ms sws_p95={:.1}ms \
encode_avg={:.1}ms encode_p95={:.1}ms total_avg={:.1}ms total_p95={:.1}ms \
output_bps={:.0} frame_bytes_p95={} frame_bytes_max={}",
self.elapsed_secs,
self.capture_fps,
self.encoded_fps,
self.sent_fps,
self.capture_frames,
self.encoded_frames,
self.sent_frames,
self.pipewire_dropped,
self.over_budget_count,
self.duplicate_frames_skipped,
self.capture_queue_depth,
self.encoded_queue_depth,
self.capture_gap_avg_ms,
self.capture_gap_p95_ms,
self.capture_gap_max_ms,
self.encoded_gap_avg_ms,
self.encoded_gap_p95_ms,
self.encoded_gap_max_ms,
self.sent_gap_avg_ms,
self.sent_gap_p95_ms,
self.sent_gap_max_ms,
self.frame_age_avg_ms,
self.frame_age_p95_ms,
self.frame_age_max_ms,
self.send_wait_p95_ms,
self.import_avg_ms,
self.import_p95_ms,
self.scale_avg_ms,
self.scale_p95_ms,
self.transfer_avg_ms,
self.transfer_p95_ms,
self.sws_avg_ms,
self.sws_p95_ms,
self.encode_avg_ms,
self.encode_p95_ms,
self.total_avg_ms,
self.total_p95_ms,
self.output_bytes_per_sec,
self.output_frame_bytes_p95,
self.output_frame_bytes_max,
)
}
+9 -311
View File
@@ -1,8 +1,12 @@
/// Coordinate transformation module for Wayland output transforms.
///
/// Handles the 8 `wl_output` transform variants (rotation + reflection)
/// and ROI clipping for screen capture.
///
//! Coordinate transformation module for Wayland output transforms.
//!
//! Historically exposed a family of `Rect`/`screen_to_frame`/`fit_inside_bounds`
//! helpers for ROI-based capture clipping. Those were never wired into the
//! capture pipeline (we capture full frames and let FFmpeg's filter graph handle
//! any scaling/rotation); they have been removed. Only `Transform` and the
//! `transpose_if_transform_transposed` helper remain — both are actively used by
//! `state.rs` and `avhw.rs`.
/// Wayland output transform enum, matching `wl_output::Transform`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Transform {
@@ -16,68 +20,6 @@ pub enum Transform {
Flipped270,
}
/// Axis-aligned rectangle in integer coordinates.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Rect {
pub x: i32,
pub y: i32,
pub w: i32,
pub h: i32,
}
/// Returns the 2×2 basis matrix (a, b, c, d) for the given transform.
///
/// The matrix represents the affine mapping from screen coordinates to
/// frame coordinates:
///
/// ```text
/// [new_x] [a b] [x]
/// [new_y] = [c d] [y]
/// ```
pub fn transform_basis(transform: Transform) -> (i32, i32, i32, i32) {
match transform {
Transform::Normal => (1, 0, 0, 1),
Transform::Normal90 => (0, 1, -1, 0),
Transform::Normal180 => (-1, 0, 0, -1),
Transform::Normal270 => (0, -1, 1, 0),
Transform::Flipped => (-1, 0, 0, 1),
Transform::Flipped90 => (0, 1, 1, 0),
Transform::Flipped180 => (1, 0, 0, -1),
Transform::Flipped270 => (0, -1, -1, 0),
}
}
/// Transform a rectangle from screen space to frame space.
///
/// Applies the 2×2 basis matrix and computes offsets so the result
/// fits within the frame dimensions `(frame_w, frame_h)`.
///
/// ```text
/// new_x = a * x + b * y + offset_x
/// new_y = c * x + d * y + offset_y
/// ```
pub fn screen_to_frame(transform: Transform, rect: Rect, frame_w: i32, frame_h: i32) -> Rect {
let (a, b, c, d) = transform_basis(transform);
// Compute the offset so that the transformed origin maps correctly.
// For transforms with negative components, we need to shift by the
// frame dimension to keep coordinates in [0, frame_w) × [0, frame_h).
let offset_x = if a + b < 0 { frame_w } else { 0 };
let offset_y = if c + d < 0 { frame_h } else { 0 };
let new_x = a * rect.x + b * rect.y + offset_x;
let new_y = c * rect.x + d * rect.y + offset_y;
let new_w = a * rect.w + b * rect.h;
let new_h = c * rect.w + d * rect.h;
Rect {
x: new_x,
y: new_y,
w: new_w.abs(),
h: new_h.abs(),
}
}
/// Swap width and height for 90° or 270° rotations.
///
/// After a quarter-turn rotation the output dimensions are transposed
@@ -93,140 +35,10 @@ pub fn transpose_if_transform_transposed(transform: Transform, w: i32, h: i32) -
}
}
/// Clip a rectangle so it stays inside `(0, 0) .. (bounds_w, bounds_h)`.
///
/// The resulting rectangle has non-negative origin and its extent does
/// not exceed the bounds.
pub fn fit_inside_bounds(rect: Rect, bounds_w: i32, bounds_h: i32) -> Rect {
let x = rect.x.clamp(0, bounds_w);
let y = rect.y.clamp(0, bounds_h);
let right = (rect.x + rect.w).min(bounds_w);
let bottom = (rect.y + rect.h).min(bounds_h);
let w = (right - x).max(0);
let h = (bottom - y).max(0);
Rect { x, y, w, h }
}
#[cfg(test)]
mod tests {
use super::*;
// ── transform_basis ───────────────────────────────────────────
#[test]
fn basis_normal_is_identity() {
assert_eq!(transform_basis(Transform::Normal), (1, 0, 0, 1));
}
#[test]
fn basis_90_cw_rotation() {
assert_eq!(transform_basis(Transform::Normal90), (0, 1, -1, 0));
}
#[test]
fn basis_180_rotation() {
assert_eq!(transform_basis(Transform::Normal180), (-1, 0, 0, -1));
}
#[test]
fn basis_270_cw_rotation() {
assert_eq!(transform_basis(Transform::Normal270), (0, -1, 1, 0));
}
#[test]
fn basis_flipped_horizontal() {
assert_eq!(transform_basis(Transform::Flipped), (-1, 0, 0, 1));
}
#[test]
fn basis_flipped_90() {
assert_eq!(transform_basis(Transform::Flipped90), (0, 1, 1, 0));
}
#[test]
fn basis_flipped_180() {
assert_eq!(transform_basis(Transform::Flipped180), (1, 0, 0, -1));
}
#[test]
fn basis_flipped_270() {
assert_eq!(transform_basis(Transform::Flipped270), (0, -1, -1, 0));
}
// ── screen_to_frame ───────────────────────────────────────────
#[test]
fn screen_to_frame_identity_unchanged() {
let rect = Rect {
x: 10,
y: 20,
w: 100,
h: 50,
};
let result = screen_to_frame(Transform::Normal, rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 10,
y: 20,
w: 100,
h: 50
}
);
}
#[test]
fn screen_to_frame_90_rotates_origin() {
// 90° CW: top-left (0,0) in screen should map to bottom-left in frame
let rect = Rect {
x: 0,
y: 0,
w: 100,
h: 50,
};
let result = screen_to_frame(Transform::Normal90, rect, 1080, 1920);
// a=0,b=1,c=-1,d=0 => offset_x=0, offset_y=1920 (c+d=-1<0)
// new_x = 0*0 + 1*0 + 0 = 0
// new_y = -1*0 + 0*0 + 1920 = 1920
assert_eq!(result.x, 0);
assert_eq!(result.y, 1920);
// w' = 0*100 + 1*50 = 50, h' = -1*100 + 0*50 = -100 -> abs=100
assert_eq!(result.w, 50);
assert_eq!(result.h, 100);
}
#[test]
fn screen_to_frame_180_rotates() {
let rect = Rect {
x: 100,
y: 200,
w: 300,
h: 400,
};
let result = screen_to_frame(Transform::Normal180, rect, 1920, 1080);
// a=-1,b=0,c=0,d=-1, offset_x=1920, offset_y=1080
assert_eq!(result.x, -100 + 1920);
assert_eq!(result.y, -200 + 1080);
assert_eq!(result.w, 300);
assert_eq!(result.h, 400);
}
#[test]
fn screen_to_frame_flipped_horizontal() {
let rect = Rect {
x: 50,
y: 30,
w: 200,
h: 100,
};
let result = screen_to_frame(Transform::Flipped, rect, 1920, 1080);
// a=-1,b=0,c=0,d=1, offset_x=1920, offset_y=0
assert_eq!(result.x, -50 + 1920);
assert_eq!(result.y, 30);
assert_eq!(result.w, 200);
assert_eq!(result.h, 100);
}
// ── transpose_if_transform_transposed ─────────────────────────
#[test]
@@ -292,118 +104,4 @@ mod tests {
(1080, 1920)
);
}
// ── fit_inside_bounds ─────────────────────────────────────────
#[test]
fn fit_inside_already_fits() {
let rect = Rect {
x: 10,
y: 20,
w: 100,
h: 50,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(result, rect);
}
#[test]
fn fit_inside_clips_right_and_bottom() {
let rect = Rect {
x: 1800,
y: 1000,
w: 200,
h: 200,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 1800,
y: 1000,
w: 120,
h: 80
}
);
}
#[test]
fn fit_inside_clips_negative_origin() {
let rect = Rect {
x: -50,
y: -30,
w: 200,
h: 200,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 0,
y: 0,
w: 150,
h: 170
}
);
}
#[test]
fn fit_inside_completely_out_of_bounds() {
let rect = Rect {
x: 2000,
y: 2000,
w: 100,
h: 100,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 1920,
y: 1080,
w: 0,
h: 0
}
);
}
#[test]
fn fit_inside_zero_size_rect() {
let rect = Rect {
x: 100,
y: 100,
w: 0,
h: 0,
};
let result = fit_inside_bounds(rect, 1920, 1080);
assert_eq!(
result,
Rect {
x: 100,
y: 100,
w: 0,
h: 0
}
);
}
#[test]
fn fit_inside_zero_bounds() {
let rect = Rect {
x: 0,
y: 0,
w: 100,
h: 100,
};
let result = fit_inside_bounds(rect, 0, 0);
assert_eq!(
result,
Rect {
x: 0,
y: 0,
w: 0,
h: 0
}
);
}
}
+3 -173
View File
@@ -17,178 +17,8 @@ use str0m::{Candidate, Event, IceConnectionState, Input, Output, Rtc, RtcConfig}
/// bursts. See issue #23.
const FORCED_KEYFRAME_MIN_INTERVAL: Duration = Duration::from_secs(1);
// ── 嵌入式 HTML 测试页面 ──────────────────────────────────────────────────
const HTML_PAGE: &str = r#"<!DOCTYPE html>
<html>
<head><title>wl-webrtc P0</title>
<style>body{background:#000;color:#fff;font-family:monospace;display:flex;flex-direction:column;align-items:center;justify-content:center;height:100vh;margin:0}
video{max-width:90vw;max-height:80vh;border:1px solid #333}
#status{margin:12px;font-size:14px;color:#aaa}
#debug{position:fixed;bottom:8px;left:8px;font-size:11px;color:#666;max-width:90vw;white-space:pre-wrap}
#stats-panel{position:fixed;top:8px;right:8px;background:rgba(0,0,0,0.7);color:#0f0;font:11px monospace;padding:6px 10px;border-radius:4px;z-index:100;pointer-events:none;max-width:90vw;white-space:pre;line-height:1.5}
</style></head>
<body>
<div id="status">Connecting...</div>
<video id="video" autoplay playsinline muted></video>
<pre id="debug"></pre>
<div id="stats-panel"></div>
<script>
const status = document.getElementById('status');
const video = document.getElementById('video');
const debug = document.getElementById('debug');
let pc = null;
const log = msg => { debug.textContent += msg + '\n'; console.log(msg); };
function preferH264(sdp) {
const lines = sdp.split('\r\n');
const h264Pts = lines
.filter(line => line.startsWith('a=rtpmap:') && line.toUpperCase().includes('H264/90000'))
.map(line => line.match(/^a=rtpmap:(\d+)/)?.[1])
.filter(Boolean);
if (h264Pts.length === 0) return sdp;
return lines.map(line => {
if (!line.startsWith('m=video ')) return line;
const parts = line.split(' ');
const header = parts.slice(0, 3);
const pts = parts.slice(3);
const preferred = h264Pts.filter(pt => pts.includes(pt));
const rest = pts.filter(pt => !preferred.includes(pt));
return [...header, ...preferred, ...rest].join(' ');
}).join('\r\n');
}
function installStatsLogger(peer) {
const panel = document.getElementById('stats-panel');
let prev = null;
const intervalSecs = 1;
setInterval(() => {
if (peer !== pc) return;
peer.getStats().then(stats => {
let rtp = null, rtt = null, codecStr = '';
let freezeCount = null, totalFreezesDuration = null;
stats.forEach(report => {
if (report.type === 'inbound-rtp' && report.kind === 'video') rtp = report;
if (report.type === 'codec' && report.mimeType && report.mimeType.includes('H264'))
codecStr = report.mimeType + ' ' + (report.payloadType || '');
// candidate-pair: feature-detect 'selected' property
if (report.type === 'candidate-pair') {
const isSel = ('selected' in report) ? report.selected : report.state === 'succeeded';
if (isSel && typeof report.currentRoundTripTime === 'number') rtt = report.currentRoundTripTime;
}
});
// Freeze stats (feature-detect)
if (rtp && typeof rtp.freezeCount !== 'undefined') {
freezeCount = rtp.freezeCount;
totalFreezesDuration = rtp.totalFreezesDuration;
}
if (!rtp) return;
const cur = {
framesDecoded: rtp.framesDecoded || 0,
framesDropped: rtp.framesDropped || 0,
framesPerSecond: rtp.framesPerSecond || 0,
packetsLost: rtp.packetsLost || 0,
jitter: rtp.jitter || 0,
bytesReceived: rtp.bytesReceived || 0,
totalDecodeTime: rtp.totalDecodeTime || 0,
jitterBufferDelay: rtp.jitterBufferDelay || 0,
jitterBufferEmittedCount: rtp.jitterBufferEmittedCount || 0,
freezeCount: freezeCount,
totalFreezesDuration: totalFreezesDuration,
rtt: rtt,
};
// Raw log to debug element (backward compat)
log('RTP-in: decoded=' + cur.framesDecoded + ' lost=' + cur.packetsLost +
' bytes=' + cur.bytesReceived + ' fps=' + cur.framesPerSecond +
(codecStr ? ' codec=' + codecStr : ''));
if (!prev) { prev = cur; return; }
// Compute deltas
const dFrames = cur.framesDecoded - prev.framesDecoded;
const dDropped = cur.framesDropped - prev.framesDropped;
const dLost = cur.packetsLost - prev.packetsLost;
const dBytes = cur.bytesReceived - prev.bytesReceived;
const dDecodeTime = cur.totalDecodeTime - prev.totalDecodeTime;
const dJitterBufDelay = cur.jitterBufferDelay - prev.jitterBufferDelay;
const dJitterBufCount = cur.jitterBufferEmittedCount - prev.jitterBufferEmittedCount;
const kbps = Math.round(dBytes * 8 / intervalSecs / 1000);
const decodeMs = dFrames > 0 ? (dDecodeTime / dFrames * 1000).toFixed(1) : '—';
const jitterBufMs = dJitterBufCount > 0 ? (dJitterBufDelay / dJitterBufCount * 1000).toFixed(1) : '—';
const jitterMs = (cur.jitter * 1000).toFixed(1);
const rttMs = cur.rtt !== null ? (cur.rtt * 1000).toFixed(1) : null;
let line = 'FPS:' + cur.framesPerSecond +
' Decoded:' + cur.framesDecoded + '(+' + dFrames + ')' +
' Dropped:' + cur.framesDropped + (dDropped > 0 ? '(+' + dDropped + ')' : '') +
' Lost:' + dLost +
' Jitter:' + jitterMs + 'ms' +
(rttMs !== null ? ' RTT:' + rttMs + 'ms' : '') +
' Decode:' + decodeMs + 'ms' +
' JBuf:' + jitterBufMs + 'ms';
if (freezeCount !== null) {
const dFreeze = cur.freezeCount - (prev.freezeCount || 0);
if (cur.freezeCount > 0 || dFreeze > 0)
line += ' Freeze:' + cur.freezeCount + '(+' + dFreeze + ')';
}
line += ' ' + kbps + 'kbps';
panel.textContent = line;
prev = cur;
}).catch(() => {});
}, intervalSecs * 1000);
}
function connect() {
if (pc) pc.close();
pc = new RTCPeerConnection();
const peer = pc;
peer.ontrack = e => {
log('ontrack: streams=' + e.streams.length + ' kind=' + e.track.kind);
video.srcObject = e.streams[0];
status.textContent = 'Track received';
};
peer.oniceconnectionstatechange = () => {
log('ICE: ' + peer.iceConnectionState);
status.textContent = 'ICE: ' + peer.iceConnectionState;
};
peer.addTransceiver('video', { direction: 'recvonly' });
installStatsLogger(peer);
peer.createOffer().then(offer => {
offer.sdp = preferH264(offer.sdp);
return peer.setLocalDescription(offer);
})
.then(() => new Promise(resolve => {
if (peer.iceGatheringState === 'complete') resolve();
else peer.onicegatheringstatechange = () => { if (peer.iceGatheringState === 'complete') resolve(); };
}))
.then(() => fetch('/sdp', { method: 'POST', body: JSON.stringify(peer.localDescription) }))
.then(r => { if (!r.ok) throw new Error('SDP exchange failed: ' + r.status); return r.json(); })
.then(answer => { if (answer.error) throw new Error(answer.error); return peer.setRemoteDescription(answer); })
.then(() => log('SDP answer set'))
.catch(e => {
status.textContent = 'Error: ' + e.message;
log('ERROR: ' + e.message + ' — retrying in 2s...');
console.error(e);
setTimeout(connect, 2000);
});
}
connect();
</script>
</body></html>"#;
mod html_page;
use html_page::HTML_PAGE;
// ── WebRTC 状态 ───────────────────────────────────────────────────────────
@@ -536,7 +366,7 @@ impl WebRtcInner {
let now = Instant::now();
let should_honor = self
.last_forced_keyframe_at
.map_or(true, |last| now.duration_since(last) >= FORCED_KEYFRAME_MIN_INTERVAL);
.is_none_or(|last| now.duration_since(last) >= FORCED_KEYFRAME_MIN_INTERVAL);
if should_honor {
self.last_forced_keyframe_at = Some(now);
self.need_keyframe = true;
+170
View File
@@ -0,0 +1,170 @@
pub(super) const HTML_PAGE: &str = r#"<!DOCTYPE html>
<html>
<head><title>wl-webrtc P0</title>
<style>body{background:#000;color:#fff;font-family:monospace;display:flex;flex-direction:column;align-items:center;justify-content:center;height:100vh;margin:0}
video{max-width:90vw;max-height:80vh;border:1px solid #333}
#status{margin:12px;font-size:14px;color:#aaa}
#debug{position:fixed;bottom:8px;left:8px;font-size:11px;color:#666;max-width:90vw;white-space:pre-wrap}
#stats-panel{position:fixed;top:8px;right:8px;background:rgba(0,0,0,0.7);color:#0f0;font:11px monospace;padding:6px 10px;border-radius:4px;z-index:100;pointer-events:none;max-width:90vw;white-space:pre;line-height:1.5}
</style></head>
<body>
<div id="status">Connecting...</div>
<video id="video" autoplay playsinline muted></video>
<pre id="debug"></pre>
<div id="stats-panel"></div>
<script>
const status = document.getElementById('status');
const video = document.getElementById('video');
const debug = document.getElementById('debug');
let pc = null;
const log = msg => { debug.textContent += msg + '\n'; console.log(msg); };
function preferH264(sdp) {
const lines = sdp.split('\r\n');
const h264Pts = lines
.filter(line => line.startsWith('a=rtpmap:') && line.toUpperCase().includes('H264/90000'))
.map(line => line.match(/^a=rtpmap:(\d+)/)?.[1])
.filter(Boolean);
if (h264Pts.length === 0) return sdp;
return lines.map(line => {
if (!line.startsWith('m=video ')) return line;
const parts = line.split(' ');
const header = parts.slice(0, 3);
const pts = parts.slice(3);
const preferred = h264Pts.filter(pt => pts.includes(pt));
const rest = pts.filter(pt => !preferred.includes(pt));
return [...header, ...preferred, ...rest].join(' ');
}).join('\r\n');
}
function installStatsLogger(peer) {
const panel = document.getElementById('stats-panel');
let prev = null;
const intervalSecs = 1;
setInterval(() => {
if (peer !== pc) return;
peer.getStats().then(stats => {
let rtp = null, rtt = null, codecStr = '';
let freezeCount = null, totalFreezesDuration = null;
stats.forEach(report => {
if (report.type === 'inbound-rtp' && report.kind === 'video') rtp = report;
if (report.type === 'codec' && report.mimeType && report.mimeType.includes('H264'))
codecStr = report.mimeType + ' ' + (report.payloadType || '');
// candidate-pair: feature-detect 'selected' property
if (report.type === 'candidate-pair') {
const isSel = ('selected' in report) ? report.selected : report.state === 'succeeded';
if (isSel && typeof report.currentRoundTripTime === 'number') rtt = report.currentRoundTripTime;
}
});
// Freeze stats (feature-detect)
if (rtp && typeof rtp.freezeCount !== 'undefined') {
freezeCount = rtp.freezeCount;
totalFreezesDuration = rtp.totalFreezesDuration;
}
if (!rtp) return;
const cur = {
framesDecoded: rtp.framesDecoded || 0,
framesDropped: rtp.framesDropped || 0,
framesPerSecond: rtp.framesPerSecond || 0,
packetsLost: rtp.packetsLost || 0,
jitter: rtp.jitter || 0,
bytesReceived: rtp.bytesReceived || 0,
totalDecodeTime: rtp.totalDecodeTime || 0,
jitterBufferDelay: rtp.jitterBufferDelay || 0,
jitterBufferEmittedCount: rtp.jitterBufferEmittedCount || 0,
freezeCount: freezeCount,
totalFreezesDuration: totalFreezesDuration,
rtt: rtt,
};
// Raw log to debug element (backward compat)
log('RTP-in: decoded=' + cur.framesDecoded + ' lost=' + cur.packetsLost +
' bytes=' + cur.bytesReceived + ' fps=' + cur.framesPerSecond +
(codecStr ? ' codec=' + codecStr : ''));
if (!prev) { prev = cur; return; }
// Compute deltas
const dFrames = cur.framesDecoded - prev.framesDecoded;
const dDropped = cur.framesDropped - prev.framesDropped;
const dLost = cur.packetsLost - prev.packetsLost;
const dBytes = cur.bytesReceived - prev.bytesReceived;
const dDecodeTime = cur.totalDecodeTime - prev.totalDecodeTime;
const dJitterBufDelay = cur.jitterBufferDelay - prev.jitterBufferDelay;
const dJitterBufCount = cur.jitterBufferEmittedCount - prev.jitterBufferEmittedCount;
const kbps = Math.round(dBytes * 8 / intervalSecs / 1000);
const decodeMs = dFrames > 0 ? (dDecodeTime / dFrames * 1000).toFixed(1) : '—';
const jitterBufMs = dJitterBufCount > 0 ? (dJitterBufDelay / dJitterBufCount * 1000).toFixed(1) : '—';
const jitterMs = (cur.jitter * 1000).toFixed(1);
const rttMs = cur.rtt !== null ? (cur.rtt * 1000).toFixed(1) : null;
let line = 'FPS:' + cur.framesPerSecond +
' Decoded:' + cur.framesDecoded + '(+' + dFrames + ')' +
' Dropped:' + cur.framesDropped + (dDropped > 0 ? '(+' + dDropped + ')' : '') +
' Lost:' + dLost +
' Jitter:' + jitterMs + 'ms' +
(rttMs !== null ? ' RTT:' + rttMs + 'ms' : '') +
' Decode:' + decodeMs + 'ms' +
' JBuf:' + jitterBufMs + 'ms';
if (freezeCount !== null) {
const dFreeze = cur.freezeCount - (prev.freezeCount || 0);
if (cur.freezeCount > 0 || dFreeze > 0)
line += ' Freeze:' + cur.freezeCount + '(+' + dFreeze + ')';
}
line += ' ' + kbps + 'kbps';
panel.textContent = line;
prev = cur;
}).catch(() => {});
}, intervalSecs * 1000);
}
function connect() {
if (pc) pc.close();
pc = new RTCPeerConnection();
const peer = pc;
peer.ontrack = e => {
log('ontrack: streams=' + e.streams.length + ' kind=' + e.track.kind);
video.srcObject = e.streams[0];
status.textContent = 'Track received';
};
peer.oniceconnectionstatechange = () => {
log('ICE: ' + peer.iceConnectionState);
status.textContent = 'ICE: ' + peer.iceConnectionState;
};
peer.addTransceiver('video', { direction: 'recvonly' });
installStatsLogger(peer);
peer.createOffer().then(offer => {
offer.sdp = preferH264(offer.sdp);
return peer.setLocalDescription(offer);
})
.then(() => new Promise(resolve => {
if (peer.iceGatheringState === 'complete') resolve();
else peer.onicegatheringstatechange = () => { if (peer.iceGatheringState === 'complete') resolve(); };
}))
.then(() => fetch('/sdp', { method: 'POST', body: JSON.stringify(peer.localDescription) }))
.then(r => { if (!r.ok) throw new Error('SDP exchange failed: ' + r.status); return r.json(); })
.then(answer => { if (answer.error) throw new Error(answer.error); return peer.setRemoteDescription(answer); })
.then(() => log('SDP answer set'))
.catch(e => {
status.textContent = 'Error: ' + e.message;
log('ERROR: ' + e.message + ' — retrying in 2s...');
console.error(e);
setTimeout(connect, 2000);
});
}
connect();
</script>
</body></html>"#;