Compare commits

...
2 Commits
Author SHA1 Message Date
dailz 0e91c793c7 docs: add AGENTS.md project guide
Capture sources of truth, build/test instructions, runtime architecture,
and unsafe/FFI notes for AI agents working in this repo.
2026-06-13 22:46:41 +08:00
dailz 3e60258627 feat(portal): BWE-driven resolution adaptation + duplicate frame skipping
WebRTC client bandwidth estimate now drives both encoder bitrate and
resolution tier selection, replacing the previous static-target encoder.

- webrtc.rs: enable str0m BWE (seeded at 5 Mbps), surface
  EgressBitrateEstimate + KeyframeRequest events, expose
  get_bwe_estimate() / set_need_keyframe()
- state_portal.rs: wire bitrate/resolution channels between the WebRTC
  thread and the encode thread; tier ladder [1440p, 1080p, 720p] with
  downscale at 60% budget and upscale hysteresis (120% sustained 10s)
- avhw.rs: SwEncImport::poll_resolution_commands() rebuilds the import
  filter graph on UpdateResolution; SwEncEncode::recreate_encoder()
  rebuilds sws/enc_video/yuv_frame atomically; hash_sampled_y_plane()
  skips duplicate frames; VBV x264opts cap IDR bursts; H.264 level 4.0
  (muxer) / 4.2 (WebRTC)
- state.rs: sync wlr-screencopy GOP to fps*2 max 20 for parity
- fix: drain bitrate_rx + resolution_rx BEFORE the stride check in
  encode_cpu_frame() so the new (smaller-stride) frame produced after
  a resolution change does not hit the stale (larger) enc_width and
  crash the encode thread
- WebRTC GOP widened to fps*2 max 20 (was fps/2 max 10)
2026-06-13 22:46:33 +08:00
10 changed files with 945 additions and 169 deletions
+40
View File
@@ -0,0 +1,40 @@
# AGENTS.md
## Sources of truth
- Trust current Rust source, `Cargo.toml`, `shell.nix`, and tests over `docs/superpowers/*` or `analysis.md`; those docs contain historical/aspirational modules that are not in the tree.
- Crate edition is 2021, not the default Rust 2024. The package exposes a library, three binaries (`wl-webrtc`, `vaapi_import_bench`, `sw_encode_bench`), and two examples (`list_globals`, `test_portal`).
## Setup and build
- Native prerequisites are FFmpeg 6+ dev libs with VAAPI, Wayland protocols/libs, libdrm, PipeWire, and libclang. `shell.nix` provides FFmpeg/Wayland/libdrm/Mesa/libva/clang and `LIBCLANG_PATH`, but does not currently list PipeWire.
- Normal build: `cargo build`. Release binary required by README and integration tests: `cargo build --release`.
- `Cargo.toml` only warns on `clippy::undocumented_unsafe_blocks`; do not assume a broader clippy policy exists unless you add one.
## Testing and verification
- Unit/focused tests can run with filters, e.g. `cargo test transform`, `cargo test fps_limit`, `cargo test backend_detect`.
- Full `cargo test` includes `tests/integration_test.rs`, which shells out to `target/release/wl-webrtc`; run `cargo build --release` first or those tests fail before reaching code behavior.
- Hardware/live tests are marked ignored; run them only on a Wayland session with VAAPI-capable GPU and writable output: `cargo test -- --ignored`.
- CLI smoke test surface: `target/release/wl-webrtc --help` and invalid-argument rejection. Real capture needs Wayland plus either wlr-screencopy or XDG Portal/PipeWire.
- The README CLI table is stale for `--backend` and `--no-persist`; `src/args.rs` is authoritative for flags.
## Runtime architecture
- `src/main.rs` is the real entrypoint: parse `Args`, initialize tracing from `RUST_LOG` or `-v`, reject non-H.264, require `--output` or `--port`, detect backend, then run one of two loops.
- Backend detection in `src/backend_detect.rs`: explicit `--backend portal|screencopy` wins; otherwise wlr-screencopy is preferred when the Wayland global `zwlr_screencopy_manager_v1` exists, else Portal/PipeWire is used if D-Bus ScreenCast is available.
- Do not use `ashpd` for backend availability checks; `backend_detect.rs` intentionally uses raw `zbus` because `ashpd` caches a `zbus::Connection` in a global and can hang after its owning Tokio runtime is dropped.
- `src/state.rs` drives the wlroots path using a mio Wayland fd loop and `State<CapWlrScreencopy>`; `src/state_portal.rs` drives the Portal/PipeWire path through `CapPortal` frame channels.
- `src/webrtc.rs` is a small embedded HTTP/WebRTC signaling server using `str0m`; `--port 0` means file-output mode, `--port > 0` enables WebRTC mode.
## Unsafe and FFI work
- FFmpeg/VAAPI/PipeWire code relies on raw FFI and many `unsafe` blocks. Preserve nearby `// SAFETY:` explanations and add one for any new unsafe block.
- `src/avhw.rs` owns FFmpeg `AVBufferRef`/frame contexts and has explicit `unsafe impl Send`; avoid moving those wrappers across threads without rechecking the documented exclusivity assumptions.
- `CapPortal` stores the portal restore token under the user cache directory (`wl-webrtc/portal-restore-token`); use `--no-persist` when manually testing fresh authorization behavior.
## Useful manual commands
- List Wayland globals: `cargo run --example list_globals`.
- Portal permission smoke test: `cargo run --example test_portal`.
- Portal/VAAPI benchmarks require a screen-share dialog and hardware: `cargo run --bin vaapi_import_bench -- --output /tmp/vaapi_bench.mp4` and `cargo run --bin sw_encode_bench -- --output /tmp/bench_test.mp4`.
+421 -21
View File
@@ -3,10 +3,10 @@ use std::mem;
use std::os::fd::{AsRawFd, RawFd}; use std::os::fd::{AsRawFd, RawFd};
use std::os::raw::c_void; use std::os::raw::c_void;
use std::path::Path; use std::path::Path;
use std::ptr;
use std::slice; use std::slice;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc; use std::sync::Arc;
use std::ptr;
use anyhow::{bail, Result}; use anyhow::{bail, Result};
use ffmpeg_next as ff; use ffmpeg_next as ff;
@@ -16,6 +16,23 @@ use ffmpeg_next::packet::Mut as _;
use crate::cap_portal::PwDmaBufFrame; use crate::cap_portal::PwDmaBufFrame;
use crate::transform::{transpose_if_transform_transposed, Transform}; use crate::transform::{transpose_if_transform_transposed, Transform};
// ---------------------------------------------------------------------------
// Bitrate feedback command (WebRTC BWE → SW encoder)
// ---------------------------------------------------------------------------
/// Commands sent from the WebRTC thread to the SW encoder when the
/// bandwidth estimate changes significantly.
pub enum BitrateCommand {
UpdateBitrate { target_bps: u64 },
UpdateResolution { width: u32, height: u32 },
}
#[derive(Clone, Copy, Debug)]
pub struct ResolutionChange {
pub width: u32,
pub height: u32,
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// AvHwDevCtx // AvHwDevCtx
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -334,7 +351,9 @@ impl EncState {
transform, transform,
)?; )?;
let mut sink_ctx = video_filter.get("out").ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?; let mut sink_ctx = video_filter
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
// SAFETY: sink_ctx is a live buffersink; the returned hw_frames_ctx is // SAFETY: sink_ctx is a live buffersink; the returned hw_frames_ctx is
// borrowed, so av_buffer_ref creates an owned reference. // borrowed, so av_buffer_ref creates an owned reference.
let sink_hw_frames = unsafe { let sink_hw_frames = unsafe {
@@ -461,7 +480,10 @@ impl EncState {
ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr()) ffi::avcodec_parameters_from_context((*stream_ptr).codecpar, enc_video.as_ptr())
}; };
if ret < 0 { if ret < 0 {
bail!("Failed to copy encoder parameters to stream: {}", ff_err(ret)); bail!(
"Failed to copy encoder parameters to stream: {}",
ff_err(ret)
);
} }
// SAFETY: Copy encoder time_base to stream. // SAFETY: Copy encoder time_base to stream.
@@ -510,9 +532,15 @@ impl EncState {
} }
pub fn encode_frame(&mut self, hw_frame: &ff::frame::Video) -> Result<()> { pub fn encode_frame(&mut self, hw_frame: &ff::frame::Video) -> Result<()> {
let mut filter_src_ctx = self.video_filter.get("in").ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?; let mut filter_src_ctx = self
.video_filter
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source(); let mut filter_src = filter_src_ctx.source();
let mut filter_sink_ctx = self.video_filter.get("out").ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?; let mut filter_sink_ctx = self
.video_filter
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink(); let mut filter_sink = filter_sink_ctx.sink();
// SAFETY: hw_frame is a valid VAAPI hardware frame from capture. // SAFETY: hw_frame is a valid VAAPI hardware frame from capture.
@@ -552,14 +580,20 @@ impl EncState {
pub fn flush(&mut self) -> Result<()> { pub fn flush(&mut self) -> Result<()> {
// Flush filter graph // Flush filter graph
let mut filter_src_ctx = self.video_filter.get("in").ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?; let mut filter_src_ctx = self
.video_filter
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source(); let mut filter_src = filter_src_ctx.source();
if let Err(e) = filter_src.flush() { if let Err(e) = filter_src.flush() {
tracing::debug!("filter source flush error: {e}"); tracing::debug!("filter source flush error: {e}");
} }
// Drain filter // Drain filter
let mut filter_sink_ctx = self.video_filter.get("out").ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?; let mut filter_sink_ctx = self
.video_filter
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink(); let mut filter_sink = filter_sink_ctx.sink();
loop { loop {
let mut filtered = ff::frame::Video::empty(); let mut filtered = ff::frame::Video::empty();
@@ -667,8 +701,13 @@ pub struct SwEncImport {
hw_dev: AvHwDevCtx, hw_dev: AvHwDevCtx,
frames_rgb: AvHwFrameCtx, frames_rgb: AvHwFrameCtx,
filter_graph: ff::filter::Graph, filter_graph: ff::filter::Graph,
width: u32,
height: u32,
enc_width: u32, enc_width: u32,
enc_height: u32, enc_height: u32,
fps: u32,
resolution_rx: Option<crossbeam_channel::Receiver<BitrateCommand>>,
encoder_resolution_tx: Option<crossbeam_channel::Sender<ResolutionChange>>,
} }
impl SwEncImport { impl SwEncImport {
@@ -698,20 +737,50 @@ impl SwEncImport {
hw_dev, hw_dev,
frames_rgb, frames_rgb,
filter_graph, filter_graph,
width,
height,
enc_width, enc_width,
enc_height, enc_height,
fps,
resolution_rx: None,
encoder_resolution_tx: None,
}) })
} }
#[allow(clippy::too_many_arguments)]
pub fn new_with_resolution_control(
drm_device: &Path,
width: u32,
height: u32,
enc_width: u32,
enc_height: u32,
fps: u32,
resolution_rx: crossbeam_channel::Receiver<BitrateCommand>,
encoder_resolution_tx: crossbeam_channel::Sender<ResolutionChange>,
) -> Result<Self> {
let mut this = Self::new(drm_device, width, height, enc_width, enc_height, fps)?;
this.resolution_rx = Some(resolution_rx);
this.encoder_resolution_tx = Some(encoder_resolution_tx);
Ok(this)
}
pub fn frames_rgb(&self) -> &AvHwFrameCtx { pub fn frames_rgb(&self) -> &AvHwFrameCtx {
let _ = self.hw_dev.as_ptr(); let _ = self.hw_dev.as_ptr();
&self.frames_rgb &self.frames_rgb
} }
pub fn import_and_scale(&mut self, hw_frame: &ff::frame::Video) -> Result<CpuNv12Frame> { pub fn import_and_scale(&mut self, hw_frame: &ff::frame::Video) -> Result<CpuNv12Frame> {
let mut filter_src_ctx = self.filter_graph.get("in").ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?; self.poll_resolution_commands()?;
let mut filter_src_ctx = self
.filter_graph
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source(); let mut filter_src = filter_src_ctx.source();
let mut filter_sink_ctx = self.filter_graph.get("out").ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?; let mut filter_sink_ctx = self
.filter_graph
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink(); let mut filter_sink = filter_sink_ctx.sink();
filter_src filter_src
@@ -740,20 +809,28 @@ impl SwEncImport {
} }
if extra_count > 0 { if extra_count > 0 {
tracing::warn!("software import filter produced {extra_count} extra frame(s); dropping extras"); tracing::warn!(
"software import filter produced {extra_count} extra frame(s); dropping extras"
);
} }
first.ok_or_else(|| anyhow::anyhow!("software pipeline produced no scaled frame")) first.ok_or_else(|| anyhow::anyhow!("software pipeline produced no scaled frame"))
} }
pub fn flush_import(&mut self) -> Result<Vec<CpuNv12Frame>> { pub fn flush_import(&mut self) -> Result<Vec<CpuNv12Frame>> {
let mut filter_src_ctx = self.filter_graph.get("in").ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?; let mut filter_src_ctx = self
.filter_graph
.get("in")
.ok_or_else(|| anyhow::anyhow!("filter 'in' not found"))?;
let mut filter_src = filter_src_ctx.source(); let mut filter_src = filter_src_ctx.source();
if let Err(e) = filter_src.flush() { if let Err(e) = filter_src.flush() {
tracing::debug!("filter source flush error: {e}"); tracing::debug!("filter source flush error: {e}");
} }
let mut filter_sink_ctx = self.filter_graph.get("out").ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?; let mut filter_sink_ctx = self
.filter_graph
.get("out")
.ok_or_else(|| anyhow::anyhow!("filter 'out' not found"))?;
let mut filter_sink = filter_sink_ctx.sink(); let mut filter_sink = filter_sink_ctx.sink();
let mut frames = Vec::new(); let mut frames = Vec::new();
loop { loop {
@@ -767,6 +844,54 @@ impl SwEncImport {
Ok(frames) Ok(frames)
} }
fn poll_resolution_commands(&mut self) -> Result<()> {
let Some(rx) = self.resolution_rx.as_ref().cloned() else {
return Ok(());
};
let mut requested = None;
while let Ok(cmd) = rx.try_recv() {
match cmd {
BitrateCommand::UpdateResolution { width, height } => {
requested = Some((width & !1, height & !1));
}
BitrateCommand::UpdateBitrate { .. } => {}
}
}
let Some((width, height)) = requested else {
return Ok(());
};
if width == self.enc_width && height == self.enc_height {
return Ok(());
}
tracing::info!(
from = format_args!("{}x{}", self.enc_width, self.enc_height),
to = format_args!("{}x{}", width, height),
"rebuilding software import filter graph for resolution change"
);
let _ = self.flush_import();
self.filter_graph = build_swenc_filter_graph(
&self.hw_dev,
&self.frames_rgb,
self.width,
self.height,
width,
height,
self.fps,
)?;
self.enc_width = width;
self.enc_height = height;
if let Some(tx) = &self.encoder_resolution_tx {
tx.send(ResolutionChange { width, height })
.map_err(|_| anyhow::anyhow!("encoder resolution channel disconnected"))?;
}
Ok(())
}
fn transfer_filtered_to_cpu(&self, filtered: &ff::frame::Video) -> Result<CpuNv12Frame> { fn transfer_filtered_to_cpu(&self, filtered: &ff::frame::Video) -> Result<CpuNv12Frame> {
// SAFETY: av_frame_alloc returns a newly allocated AVFrame or null, // SAFETY: av_frame_alloc returns a newly allocated AVFrame or null,
// which is checked below. // which is checked below.
@@ -798,7 +923,9 @@ impl SwEncImport {
} }
let y_stride = (*sw_nv12).linesize[0] as usize; let y_stride = (*sw_nv12).linesize[0] as usize;
let uv_stride = (*sw_nv12).linesize[1] as usize; let uv_stride = (*sw_nv12).linesize[1] as usize;
if (*sw_nv12).width != self.enc_width as i32 || (*sw_nv12).height != self.enc_height as i32 { if (*sw_nv12).width != self.enc_width as i32
|| (*sw_nv12).height != self.enc_height as i32
{
ffi::av_frame_free(&mut sw_nv12); ffi::av_frame_free(&mut sw_nv12);
bail!("NV12 transfer frame has unexpected dimensions"); bail!("NV12 transfer frame has unexpected dimensions");
} }
@@ -826,12 +953,38 @@ pub struct SwEncEncode {
enc_video: ff::codec::encoder::video::Video, enc_video: ff::codec::encoder::video::Video,
output: Option<FrameOutput>, output: Option<FrameOutput>,
yuv_frame: *mut ffi::AVFrame, yuv_frame: *mut ffi::AVFrame,
last_frame_hash: u64,
frame_count: u64,
starting_timestamp: Option<i64>, starting_timestamp: Option<i64>,
frames_written: bool, frames_written: bool,
webrtc_disconnected: bool, webrtc_disconnected: bool,
webrtc_paused: Option<Arc<AtomicBool>>, webrtc_paused: Option<Arc<AtomicBool>>,
bitrate_rx: crossbeam_channel::Receiver<BitrateCommand>,
resolution_rx: crossbeam_channel::Receiver<ResolutionChange>,
enc_width: u32, enc_width: u32,
enc_height: u32, enc_height: u32,
fps: u32,
bitrate: u64,
gop_size: u32,
}
const FNV1A_OFFSET_BASIS: u64 = 0xcbf29ce484222325;
const FNV1A_PRIME: u64 = 0x100000001b3;
const Y_PLANE_HASH_ROW_STEP: usize = 8;
fn hash_sampled_y_plane(y_data: &[u8], width: usize, height: usize, stride: usize) -> u64 {
let mut hash = FNV1A_OFFSET_BASIS;
for row in (0..height).step_by(Y_PLANE_HASH_ROW_STEP) {
let row_start = row * stride;
let row_end = row_start + width;
for &byte in &y_data[row_start..row_end] {
hash ^= u64::from(byte);
hash = hash.wrapping_mul(FNV1A_PRIME);
}
}
hash
} }
// SAFETY: SwEncEncode owns sws_ctx/yuv_frame/enc_video exclusively after construction. // SAFETY: SwEncEncode owns sws_ctx/yuv_frame/enc_video exclusively after construction.
@@ -852,18 +1005,29 @@ impl SwEncEncode {
let (enc_video, octx) = let (enc_video, octx) =
create_software_h264_muxer(output_path, enc_width, enc_height, fps, bitrate, gop_size)?; create_software_h264_muxer(output_path, enc_width, enc_height, fps, bitrate, gop_size)?;
let yuv_frame = alloc_yuv420p_frame(enc_width, enc_height)?; let yuv_frame = alloc_yuv420p_frame(enc_width, enc_height)?;
let (dummy_tx, bitrate_rx) = crossbeam_channel::bounded(1);
drop(dummy_tx);
let (dummy_resolution_tx, resolution_rx) = crossbeam_channel::bounded(1);
drop(dummy_resolution_tx);
Ok(Self { Ok(Self {
sws_ctx, sws_ctx,
enc_video, enc_video,
output: Some(FrameOutput::Muxer(octx)), output: Some(FrameOutput::Muxer(octx)),
yuv_frame, yuv_frame,
last_frame_hash: 0,
frame_count: 0,
starting_timestamp: None, starting_timestamp: None,
frames_written: false, frames_written: false,
webrtc_disconnected: false, webrtc_disconnected: false,
webrtc_paused: None, webrtc_paused: None,
bitrate_rx,
resolution_rx,
enc_width, enc_width,
enc_height, enc_height,
fps,
bitrate,
gop_size,
}) })
} }
@@ -876,9 +1040,12 @@ impl SwEncEncode {
gop_size: u32, gop_size: u32,
tx: crossbeam_channel::Sender<Vec<u8>>, tx: crossbeam_channel::Sender<Vec<u8>>,
webrtc_paused: Arc<AtomicBool>, webrtc_paused: Arc<AtomicBool>,
bitrate_rx: crossbeam_channel::Receiver<BitrateCommand>,
resolution_rx: crossbeam_channel::Receiver<ResolutionChange>,
) -> Result<Self> { ) -> Result<Self> {
let sws_ctx = create_nv12_to_yuv420p_sws(enc_width, enc_height)?; let sws_ctx = create_nv12_to_yuv420p_sws(enc_width, enc_height)?;
let enc_video = create_software_h264_encoder(enc_width, enc_height, fps, bitrate, gop_size)?; let enc_video =
create_software_h264_encoder(enc_width, enc_height, fps, bitrate, gop_size)?;
let yuv_frame = alloc_yuv420p_frame(enc_width, enc_height)?; let yuv_frame = alloc_yuv420p_frame(enc_width, enc_height)?;
Ok(Self { Ok(Self {
@@ -886,12 +1053,19 @@ impl SwEncEncode {
enc_video, enc_video,
output: Some(FrameOutput::Channel(tx)), output: Some(FrameOutput::Channel(tx)),
yuv_frame, yuv_frame,
last_frame_hash: 0,
frame_count: 0,
starting_timestamp: None, starting_timestamp: None,
frames_written: false, frames_written: false,
webrtc_disconnected: false, webrtc_disconnected: false,
webrtc_paused: Some(webrtc_paused), webrtc_paused: Some(webrtc_paused),
bitrate_rx,
resolution_rx,
enc_width, enc_width,
enc_height, enc_height,
fps,
bitrate,
gop_size,
}) })
} }
@@ -914,6 +1088,28 @@ impl SwEncEncode {
if self.webrtc_disconnected { if self.webrtc_disconnected {
return Ok(()); return Ok(());
} }
// Must drain before the stride check: the import thread emits
// ResolutionChange before the new (smaller-stride) frame arrives.
while let Ok(cmd) = self.bitrate_rx.try_recv() {
match cmd {
BitrateCommand::UpdateBitrate { target_bps } => {
tracing::info!(target_bps, "updating encoder bitrate from BWE feedback");
self.bitrate = target_bps;
// SAFETY: enc_video is an opened AVCodecContext exclusively owned by &mut self.
unsafe {
let ctx = self.enc_video.as_mut_ptr();
(*ctx).bit_rate = target_bps as i64;
}
}
BitrateCommand::UpdateResolution { .. } => {}
}
}
while let Ok(change) = self.resolution_rx.try_recv() {
self.recreate_encoder(change.width, change.height)?;
}
if frame.y_stride < self.enc_width as usize || frame.uv_stride < self.enc_width as usize { if frame.y_stride < self.enc_width as usize || frame.uv_stride < self.enc_width as usize {
bail!("CPU NV12 frame stride is smaller than encoder width"); bail!("CPU NV12 frame stride is smaller than encoder width");
} }
@@ -923,6 +1119,24 @@ impl SwEncEncode {
} }
} }
let width = self.enc_width as usize;
let height = self.enc_height as usize;
let required_y_len = frame.y_stride * height.saturating_sub(1) + width;
if frame.y_data.len() < required_y_len {
bail!("CPU NV12 frame Y plane is smaller than encoder dimensions");
}
let frame_index = self.frame_count;
self.frame_count = self.frame_count.saturating_add(1);
let current_hash = hash_sampled_y_plane(&frame.y_data, width, height, frame.y_stride);
let force_gop_frame = self.gop_size > 0 && frame_index % u64::from(self.gop_size) == 0;
if frame_index > 0 && !force_gop_frame && current_hash == self.last_frame_hash {
tracing::debug!(frame_index, "skipping duplicate frame");
self.last_frame_hash = current_hash;
return Ok(());
}
self.last_frame_hash = current_hash;
// SAFETY: yuv_frame is an owned reusable YUV420P frame at the same dimensions as sw_nv12; // SAFETY: yuv_frame is an owned reusable YUV420P frame at the same dimensions as sw_nv12;
// sws_ctx was created for NV12 -> YUV420P with no resize, so sws_scale only converts format. // sws_ctx was created for NV12 -> YUV420P with no resize, so sws_scale only converts format.
unsafe { unsafe {
@@ -930,7 +1144,12 @@ impl SwEncEncode {
if ret < 0 { if ret < 0 {
bail!("av_frame_make_writable failed: {}", ff_err(ret)); bail!("av_frame_make_writable failed: {}", ff_err(ret));
} }
let src_slices = [frame.y_data.as_ptr(), frame.uv_data.as_ptr(), ptr::null(), ptr::null()]; let src_slices = [
frame.y_data.as_ptr(),
frame.uv_data.as_ptr(),
ptr::null(),
ptr::null(),
];
let src_strides = [frame.y_stride as i32, frame.uv_stride as i32, 0, 0]; let src_strides = [frame.y_stride as i32, frame.uv_stride as i32, 0, 0];
let scaled = ffi::sws_scale( let scaled = ffi::sws_scale(
self.sws_ctx, self.sws_ctx,
@@ -957,13 +1176,49 @@ impl SwEncEncode {
(*self.yuv_frame).pts = pts; (*self.yuv_frame).pts = pts;
let ret = ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), self.yuv_frame); let ret = ffi::avcodec_send_frame(self.enc_video.as_mut_ptr(), self.yuv_frame);
if ret < 0 { if ret < 0 {
bail!("avcodec_send_frame failed for software encoder: {}", ff_err(ret)); bail!(
"avcodec_send_frame failed for software encoder: {}",
ff_err(ret)
);
} }
} }
self.drain_encoder(start_ts) self.drain_encoder(start_ts)
} }
fn recreate_encoder(&mut self, width: u32, height: u32) -> Result<()> {
if width == self.enc_width && height == self.enc_height {
return Ok(());
}
tracing::info!(
from = format_args!("{}x{}", self.enc_width, self.enc_height),
to = format_args!("{}x{}", width, height),
"recreating WebRTC software encoder for resolution change"
);
if !self.sws_ctx.is_null() {
// SAFETY: sws_ctx is owned exclusively by self and will be replaced below.
unsafe { ffi::sws_freeContext(self.sws_ctx) };
self.sws_ctx = ptr::null_mut();
}
if !self.yuv_frame.is_null() {
// SAFETY: yuv_frame is owned exclusively by self and will be replaced below.
unsafe { ffi::av_frame_free(&mut self.yuv_frame) };
}
self.sws_ctx = create_nv12_to_yuv420p_sws(width, height)?;
self.enc_video =
create_software_h264_encoder(width, height, self.fps, self.bitrate, self.gop_size)?;
self.yuv_frame = alloc_yuv420p_frame(width, height)?;
self.enc_width = width;
self.enc_height = height;
self.last_frame_hash = 0;
self.frame_count = 0;
Ok(())
}
fn write_trailer_if_needed(&mut self) -> Result<()> { fn write_trailer_if_needed(&mut self) -> Result<()> {
if self.frames_written { if self.frames_written {
if let Some(FrameOutput::Muxer(ref mut octx)) = self.output { if let Some(FrameOutput::Muxer(ref mut octx)) = self.output {
@@ -1026,9 +1281,8 @@ impl SwEncEncode {
// `data` is non-null, so `data` points to `size` initialized // `data` is non-null, so `data` points to `size` initialized
// bytes owned by the packet. `u8` has alignment 1, and the // bytes owned by the packet. `u8` has alignment 1, and the
// slice is copied into a Vec before the packet is unreffed. // slice is copied into a Vec before the packet is unreffed.
let data: &[u8] = unsafe { let data: &[u8] =
std::slice::from_raw_parts(raw.data, raw.size as usize) unsafe { std::slice::from_raw_parts(raw.data, raw.size as usize) };
};
match tx.try_send(data.to_vec()) { match tx.try_send(data.to_vec()) {
Ok(()) => {} Ok(()) => {}
Err(crossbeam_channel::TrySendError::Full(frame)) => { Err(crossbeam_channel::TrySendError::Full(frame)) => {
@@ -1095,7 +1349,8 @@ impl SwEncState {
"SwEncState::new: GPU downscale {width}x{height} BGRA -> {enc_width}x{enc_height} NV12, software H.264" "SwEncState::new: GPU downscale {width}x{height} BGRA -> {enc_width}x{enc_height} NV12, software H.264"
); );
let import = SwEncImport::new(drm_device, width, height, enc_width, enc_height, fps)?; let import = SwEncImport::new(drm_device, width, height, enc_width, enc_height, fps)?;
let encode = SwEncEncode::new_muxer(output_path, enc_width, enc_height, fps, bitrate, gop_size)?; let encode =
SwEncEncode::new_muxer(output_path, enc_width, enc_height, fps, bitrate, gop_size)?;
Ok(Self { import, encode }) Ok(Self { import, encode })
} }
@@ -1116,6 +1371,10 @@ impl SwEncState {
"SwEncState::new_webrtc: GPU downscale {width}x{height} BGRA -> {enc_width}x{enc_height} NV12, software H.264 -> WebRTC" "SwEncState::new_webrtc: GPU downscale {width}x{height} BGRA -> {enc_width}x{enc_height} NV12, software H.264 -> WebRTC"
); );
let import = SwEncImport::new(drm_device, width, height, enc_width, enc_height, fps)?; let import = SwEncImport::new(drm_device, width, height, enc_width, enc_height, fps)?;
let (dummy_tx, bitrate_rx) = crossbeam_channel::bounded(1);
drop(dummy_tx);
let (dummy_resolution_tx, resolution_rx) = crossbeam_channel::bounded(1);
drop(dummy_resolution_tx);
let encode = SwEncEncode::new_webrtc( let encode = SwEncEncode::new_webrtc(
enc_width, enc_width,
enc_height, enc_height,
@@ -1124,6 +1383,8 @@ impl SwEncState {
gop_size, gop_size,
tx, tx,
webrtc_paused, webrtc_paused,
bitrate_rx,
resolution_rx,
)?; )?;
Ok(Self { import, encode }) Ok(Self { import, encode })
} }
@@ -1348,6 +1609,10 @@ fn create_software_h264_muxer(
let val = CString::new("6").unwrap(); let val = CString::new("6").unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0); ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
(*enc.as_mut_ptr()).profile = ffi::AV_PROFILE_H264_HIGH as i32; (*enc.as_mut_ptr()).profile = ffi::AV_PROFILE_H264_HIGH as i32;
// SAFETY: enc is a valid, initialized AVCodecContext from
// avcodec_alloc_context3. Setting level is a simple i32 field
// assignment on a properly aligned struct.
(*enc.as_mut_ptr()).level = 40; // H.264 Level 4.0 (up to 1080p@30)
} }
} }
@@ -1472,8 +1737,17 @@ fn create_software_h264_encoder(
// High profile via AVCodecContext.profile (not x264opts — x264 rejects it there). // High profile via AVCodecContext.profile (not x264opts — x264 rejects it there).
// High enables CABAC + 8x8dct automatically. // High enables CABAC + 8x8dct automatically.
(*enc.as_mut_ptr()).profile = ffi::AV_PROFILE_H264_HIGH as i32; (*enc.as_mut_ptr()).profile = ffi::AV_PROFILE_H264_HIGH as i32;
// SAFETY: enc is a valid, initialized AVCodecContext from
// avcodec_alloc_context3. Setting level is a simple i32 field
// assignment on a properly aligned struct.
(*enc.as_mut_ptr()).level = 42; // H.264 Level 4.2 (up to 1440p@30)
let key = CString::new("x264opts").unwrap(); let key = CString::new("x264opts").unwrap();
let val = CString::new("repeat_headers=1").unwrap(); let vbv_maxrate = bitrate;
let vbv_bufsize = bitrate / 4;
let val = CString::new(format!(
"repeat_headers=1:vbv-maxrate={vbv_maxrate}:vbv-bufsize={vbv_bufsize}"
))
.unwrap();
ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0); ffi::av_opt_set((*enc.as_mut_ptr()).priv_data, key.as_ptr(), val.as_ptr(), 0);
} }
} }
@@ -1590,3 +1864,129 @@ fn build_filter_graph(
Ok(graph) Ok(graph)
} }
#[cfg(test)]
mod tests {
use super::*;
// ── Task 1: VBV x264opts formatting ──
#[test]
fn vbv_x264opts_format() {
let bitrate: u64 = 5_000_000;
let vbv_maxrate = bitrate;
let vbv_bufsize = bitrate / 4;
let opts = format!("repeat_headers=1:vbv-maxrate={vbv_maxrate}:vbv-bufsize={vbv_bufsize}");
assert!(opts.contains("vbv-maxrate=5000000"));
assert!(opts.contains("vbv-bufsize=1250000"));
}
#[test]
fn vbv_bufsize_is_quarter_of_maxrate() {
for bitrate in [1_000_000, 5_000_000, 10_000_000] {
let maxrate = bitrate;
let bufsize = bitrate / 4;
assert_eq!(bufsize * 4, maxrate, "bufsize should be maxrate/4");
}
}
// ── Task 3: GOP formula ──
#[test]
fn webrtc_gop_formula() {
assert_eq!((15u32 * 2).max(20), 30); // 15fps -> 30
assert_eq!((30u32 * 2).max(20), 60); // 30fps -> 60
assert_eq!((60u32 * 2).max(20), 120); // 60fps -> 120
assert_eq!((5u32 * 2).max(20), 20); // 5fps -> 20 (floor)
}
#[test]
fn h264_level_values() {
// Level 4.0 supports up to 1080p@30fps (used for file muxer)
assert_eq!(40i32, 40);
// Level 4.2 supports up to 1440p@30fps (used for WebRTC low-latency encoder)
assert_eq!(42i32, 42);
}
// ── Task 4: Duplicate frame hash detection ──
#[test]
fn hash_sampled_y_plane_first_frame_consistent() {
let width = 64;
let height = 64;
let stride = 64;
let y_data = vec![0u8; stride * height];
let hash1 = hash_sampled_y_plane(&y_data, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data, width, height, stride);
assert_eq!(hash1, hash2, "same input should produce same hash");
}
#[test]
fn hash_sampled_y_plane_detects_different_frames() {
let width = 64;
let height = 64;
let stride = 64;
let y_data1 = vec![0u8; stride * height];
let y_data2 = vec![128u8; stride * height];
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_ne!(
hash1, hash2,
"different frame data should produce different hashes"
);
}
#[test]
fn hash_sampled_y_plane_samples_every_8th_row() {
// Changing a non-sampled row (e.g., row 1) should NOT change the hash
let width = 64;
let height = 64;
let stride = 64;
let y_data1 = vec![0u8; stride * height];
let mut y_data2 = vec![0u8; stride * height];
// Row 1 is NOT sampled (sampling is every 8th row: 0, 8, 16, ...)
y_data2[stride * 1..stride * 1 + width].fill(255);
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_eq!(
hash1, hash2,
"non-sampled row change should not affect hash"
);
}
#[test]
fn hash_sampled_y_plane_sensitive_to_sampled_row() {
// Changing a sampled row (row 0) SHOULD change the hash
let width = 64;
let height = 64;
let stride = 64;
let y_data1 = vec![0u8; stride * height];
let mut y_data2 = vec![0u8; stride * height];
// Row 0 IS sampled (every 8th row starting from 0)
y_data2[stride * 0..stride * 0 + width].fill(255);
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_ne!(
hash1, hash2,
"sampled row change should produce different hash"
);
}
#[test]
fn hash_sampled_y_plane_handles_stride_greater_than_width() {
// Stride can be larger than width due to alignment; unused padding should not affect hash
let width = 32;
let height = 16;
let stride = 64; // padded stride
let y_data1 = vec![0u8; stride * height];
let mut y_data2 = vec![0u8; stride * height];
// Fill the padding area (columns 32..63) of row 0 with garbage
y_data2[width..stride].fill(0xFF);
let hash1 = hash_sampled_y_plane(&y_data1, width, height, stride);
let hash2 = hash_sampled_y_plane(&y_data2, width, height, stride);
assert_eq!(
hash1, hash2,
"padding bytes beyond width should not affect hash"
);
}
}
+47 -18
View File
@@ -225,9 +225,7 @@ impl CapPortal {
proxy proxy
.select_sources(&session, options) .select_sources(&session, options)
.await .await
.map_err(|e| { .map_err(|e| anyhow::anyhow!("Screen sharing permission denied: {e}"))?;
anyhow::anyhow!("Screen sharing permission denied: {e}")
})?;
let response = proxy let response = proxy
.start(&session, None, Default::default()) .start(&session, None, Default::default())
@@ -272,7 +270,10 @@ fn verify_secure_dir(path: &std::path::Path) -> bool {
match std::fs::symlink_metadata(path) { match std::fs::symlink_metadata(path) {
Ok(meta) => { Ok(meta) => {
if meta.file_type().is_symlink() { if meta.file_type().is_symlink() {
tracing::warn!("Token parent dir is a symlink, rejecting: {}", path.display()); tracing::warn!(
"Token parent dir is a symlink, rejecting: {}",
path.display()
);
return false; return false;
} }
// Must be a directory // Must be a directory
@@ -282,7 +283,10 @@ fn verify_secure_dir(path: &std::path::Path) -> bool {
} }
// Must be owned by current user // Must be owned by current user
if meta.uid() != unsafe { libc::getuid() } { if meta.uid() != unsafe { libc::getuid() } {
tracing::warn!("Token parent dir not owned by current user: {}", path.display()); tracing::warn!(
"Token parent dir not owned by current user: {}",
path.display()
);
return false; return false;
} }
// No group or other permissions (mode must be 0o700 exactly within the 0o777 mask) // No group or other permissions (mode must be 0o700 exactly within the 0o777 mask)
@@ -346,7 +350,10 @@ fn load_restore_token_from(path: PathBuf) -> Option<String> {
}; };
if meta.file_type().is_symlink() { if meta.file_type().is_symlink() {
tracing::warn!("Token file is a symlink, refusing to read: {}", path.display()); tracing::warn!(
"Token file is a symlink, refusing to read: {}",
path.display()
);
return None; return None;
} }
if !meta.is_file() { if !meta.is_file() {
@@ -369,7 +376,11 @@ fn load_restore_token_from(path: PathBuf) -> Option<String> {
let token = std::fs::read_to_string(&path).ok()?; let token = std::fs::read_to_string(&path).ok()?;
let trimmed = token.trim().to_string(); let trimmed = token.trim().to_string();
if trimmed.is_empty() { None } else { Some(trimmed) } if trimmed.is_empty() {
None
} else {
Some(trimmed)
}
} }
fn save_restore_token(token: &str) { fn save_restore_token(token: &str) {
@@ -490,7 +501,9 @@ fn pipewire_thread(ctx: PwThreadCtx) {
let mainloop = match pw::main_loop::MainLoopBox::new(None) { let mainloop = match pw::main_loop::MainLoopBox::new(None) {
Ok(ml) => ml, Ok(ml) => ml,
Err(e) => { Err(e) => {
if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("MainLoop::new failed: {e}"))) { if let Err(e) =
event_tx.try_send(PwCtrlEvent::Error(format!("MainLoop::new failed: {e}")))
{
tracing::error!("MainLoop::new failed and error channel also failed: {e}"); tracing::error!("MainLoop::new failed and error channel also failed: {e}");
} }
return; return;
@@ -500,7 +513,9 @@ fn pipewire_thread(ctx: PwThreadCtx) {
let context = match pw::context::ContextBox::new(mainloop.loop_(), None) { let context = match pw::context::ContextBox::new(mainloop.loop_(), None) {
Ok(c) => c, Ok(c) => c,
Err(e) => { Err(e) => {
if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("Context::new failed: {e}"))) { if let Err(e) =
event_tx.try_send(PwCtrlEvent::Error(format!("Context::new failed: {e}")))
{
tracing::error!("Context::new failed and error channel also failed: {e}"); tracing::error!("Context::new failed and error channel also failed: {e}");
} }
return; return;
@@ -510,7 +525,8 @@ fn pipewire_thread(ctx: PwThreadCtx) {
let core = match context.connect_fd(pw_fd, None) { let core = match context.connect_fd(pw_fd, None) {
Ok(c) => c, Ok(c) => c,
Err(e) => { Err(e) => {
if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("connect_fd failed: {e}"))) { if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("connect_fd failed: {e}")))
{
tracing::error!("connect_fd failed and error channel also failed: {e}"); tracing::error!("connect_fd failed and error channel also failed: {e}");
} }
return; return;
@@ -534,7 +550,9 @@ fn pipewire_thread(ctx: PwThreadCtx) {
) { ) {
Ok(s) => s, Ok(s) => s,
Err(e) => { Err(e) => {
if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("Stream::new failed: {e}"))) { if let Err(e) =
event_tx.try_send(PwCtrlEvent::Error(format!("Stream::new failed: {e}")))
{
tracing::error!("Stream::new failed and error channel also failed: {e}"); tracing::error!("Stream::new failed and error channel also failed: {e}");
} }
return; return;
@@ -608,8 +626,10 @@ fn pipewire_thread(ctx: PwThreadCtx) {
"PipeWire format negotiated: {width}x{height}, \ "PipeWire format negotiated: {width}x{height}, \
drm_format={drm_format:#010x}, modifier={modifier:#x}, \ drm_format={drm_format:#010x}, modifier={modifier:#x}, \
framerate={}/{}, max_framerate={}/{}", framerate={}/{}, max_framerate={}/{}",
framerate.num, framerate.denom, framerate.num,
max_framerate.num, max_framerate.denom, framerate.denom,
max_framerate.num,
max_framerate.denom,
); );
} }
}) })
@@ -621,7 +641,6 @@ fn pipewire_thread(ctx: PwThreadCtx) {
let frame_tx = frame_tx.clone(); let frame_tx = frame_tx.clone();
let dropped = dropped; let dropped = dropped;
move |stream, _| { move |stream, _| {
let raw_buf = unsafe { stream.dequeue_raw_buffer() }; let raw_buf = unsafe { stream.dequeue_raw_buffer() };
if raw_buf.is_null() { if raw_buf.is_null() {
tracing::trace!("process: null raw_buf"); tracing::trace!("process: null raw_buf");
@@ -742,7 +761,8 @@ fn pipewire_thread(ctx: PwThreadCtx) {
StreamFlags::AUTOCONNECT | StreamFlags::MAP_BUFFERS, StreamFlags::AUTOCONNECT | StreamFlags::MAP_BUFFERS,
&mut params, &mut params,
) { ) {
if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("stream.connect failed: {e}"))) { if let Err(e) = event_tx.try_send(PwCtrlEvent::Error(format!("stream.connect failed: {e}")))
{
tracing::error!("stream.connect failed and error channel also failed: {e}"); tracing::error!("stream.connect failed and error channel also failed: {e}");
} }
return; return;
@@ -918,7 +938,10 @@ mod tests {
let meta = std::fs::symlink_metadata(&new_dir).unwrap(); let meta = std::fs::symlink_metadata(&new_dir).unwrap();
let mode = meta.permissions().mode() & 0o777; let mode = meta.permissions().mode() & 0o777;
assert_eq!(mode, 0o700, "created directory should be 0700, got {mode:o}"); assert_eq!(
mode, 0o700,
"created directory should be 0700, got {mode:o}"
);
} }
#[test] #[test]
@@ -933,7 +956,10 @@ mod tests {
let meta = std::fs::symlink_metadata(path).unwrap(); let meta = std::fs::symlink_metadata(path).unwrap();
let mode = meta.permissions().mode() & 0o777; let mode = meta.permissions().mode() & 0o777;
assert_eq!(mode, 0o700, "tightened directory should be 0700, got {mode:o}"); assert_eq!(
mode, 0o700,
"tightened directory should be 0700, got {mode:o}"
);
} }
#[test] #[test]
@@ -947,7 +973,10 @@ mod tests {
let meta = std::fs::symlink_metadata(&token_path).unwrap(); let meta = std::fs::symlink_metadata(&token_path).unwrap();
let mode = meta.permissions().mode() & 0o777; let mode = meta.permissions().mode() & 0o777;
assert_eq!(mode, 0o600, "token file should be 0600, got {mode:o}"); assert_eq!(mode, 0o600, "token file should be 0600, got {mode:o}");
assert_eq!(std::fs::read_to_string(&token_path).unwrap(), "secret-token-123"); assert_eq!(
std::fs::read_to_string(&token_path).unwrap(),
"secret-token-123"
);
} }
#[test] #[test]
+6 -1
View File
@@ -82,7 +82,12 @@ mod tests {
} }
} }
assert!(outputs.len() >= 3, "expected at least 3 outputs, got {} ({:?})", outputs.len(), outputs); assert!(
outputs.len() >= 3,
"expected at least 3 outputs, got {} ({:?})",
outputs.len(),
outputs
);
assert_eq!(outputs[0], 0); assert_eq!(outputs[0], 0);
} }
+1 -1
View File
@@ -4,8 +4,8 @@ pub mod backend_detect;
pub mod cap_portal; pub mod cap_portal;
pub mod cap_wlr_screencopy; pub mod cap_wlr_screencopy;
pub mod fps_limit; pub mod fps_limit;
pub mod stats;
pub mod state; pub mod state;
pub mod state_portal; pub mod state_portal;
pub mod stats;
pub mod transform; pub mod transform;
pub mod webrtc; pub mod webrtc;
+13 -5
View File
@@ -15,9 +15,9 @@ mod backend_detect; // 截屏后端自动检测(wlroots vs Portal/PipeWire
mod cap_portal; // XDG Portal 屏幕捕获 mod cap_portal; // XDG Portal 屏幕捕获
mod cap_wlr_screencopy; // wlroots wlr-screencopy 截屏协议 mod cap_wlr_screencopy; // wlroots wlr-screencopy 截屏协议
mod fps_limit; // 帧率限制器 mod fps_limit; // 帧率限制器
mod stats; // 管道性能统计(卡顿诊断)
mod state; // wlr-screencopy 后端的主状态机 mod state; // wlr-screencopy 后端的主状态机
mod state_portal; // Portal/PipeWire 后端的主状态机 mod state_portal; // Portal/PipeWire 后端的主状态机
mod stats; // 管道性能统计(卡顿诊断)
mod transform; // 图像变换(旋转/翻转) mod transform; // 图像变换(旋转/翻转)
mod webrtc; // WebRTC 传输(str0m Sans-IO mod webrtc; // WebRTC 传输(str0m Sans-IO
@@ -46,8 +46,7 @@ fn main() -> Result<()> {
// 根据 verbose 模式或 RUST_LOG 环境变量设置日志级别 // 根据 verbose 模式或 RUST_LOG 环境变量设置日志级别
// 支持 RUST_LOG 粒度控制(如 RUST_LOG=wl_webrtc::webrtc=trace // 支持 RUST_LOG 粒度控制(如 RUST_LOG=wl_webrtc::webrtc=trace
let env_filter = tracing_subscriber::EnvFilter::try_from_default_env() let env_filter = tracing_subscriber::EnvFilter::try_from_default_env().unwrap_or_else(|_| {
.unwrap_or_else(|_| {
if args.verbose { if args.verbose {
tracing_subscriber::EnvFilter::new("debug") tracing_subscriber::EnvFilter::new("debug")
} else { } else {
@@ -60,7 +59,14 @@ fn main() -> Result<()> {
.init(); .init();
tracing::info!("wl-webrtc starting"); tracing::info!("wl-webrtc starting");
tracing::debug!("Args: output={:?} fps={} codec={} port={} verbose={}", args.output, args.fps, args.codec, args.port, args.verbose); tracing::debug!(
"Args: output={:?} fps={} codec={} port={} verbose={}",
args.output,
args.fps,
args.codec,
args.port,
args.verbose
);
// MVP 阶段仅支持 H.264 编码,不支持 HEVC // MVP 阶段仅支持 H.264 编码,不支持 HEVC
if args.codec != "h264" { if args.codec != "h264" {
@@ -352,7 +358,9 @@ fn run_portal_pipewire(args: Args) -> Result<()> {
// Portal 状态机遇到致命错误时退出 // Portal 状态机遇到致命错误时退出
if state.is_errored() { if state.is_errored() {
tracing::error!("Fatal error in portal state machine (check preceding error logs), exiting"); tracing::error!(
"Fatal error in portal state machine (check preceding error logs), exiting"
);
running = false; running = false;
} }
} }
+31 -13
View File
@@ -539,7 +539,10 @@ impl<S: CaptureSource> State<S> {
// takes ownership of the fd, and the original fd is owned by map_frame. // takes ownership of the fd, and the original fd is owned by map_frame.
let fd_dup = unsafe { libc::dup(obj.fd) }; let fd_dup = unsafe { libc::dup(obj.fd) };
if fd_dup < 0 { if fd_dup < 0 {
tracing::error!("failed to dup dma-buf fd: {}", std::io::Error::last_os_error()); tracing::error!(
"failed to dup dma-buf fd: {}",
std::io::Error::last_os_error()
);
// wayland-client does not auto-destroy params on Drop. // wayland-client does not auto-destroy params on Drop.
params.destroy(); params.destroy();
self.errored = true; self.errored = true;
@@ -641,7 +644,11 @@ impl<S: CaptureSource> State<S> {
if last.elapsed() >= std::time::Duration::from_secs(10) { if last.elapsed() >= std::time::Duration::from_secs(10) {
let delta = self.stats_frames; let delta = self.stats_frames;
let fps = delta as f64 / last.elapsed().as_secs_f64(); let fps = delta as f64 / last.elapsed().as_secs_f64();
tracing::info!(frames = self.stats_frames, fps = format!("{fps:.1}"), "encoding stats"); tracing::info!(
frames = self.stats_frames,
fps = format!("{fps:.1}"),
"encoding stats"
);
self.stats_last_time = Some(std::time::Instant::now()); self.stats_last_time = Some(std::time::Instant::now());
self.stats_frames = 0; self.stats_frames = 0;
} }
@@ -672,7 +679,9 @@ impl<S: CaptureSource> State<S> {
} }
pub fn poll_webrtc(&mut self) -> Result<()> { pub fn poll_webrtc(&mut self) -> Result<()> {
let Some(ref mut wrtc) = self.webrtc else { return Ok(()) }; let Some(ref mut wrtc) = self.webrtc else {
return Ok(());
};
wrtc.handle_signaling()?; wrtc.handle_signaling()?;
wrtc.poll_and_feed()?; wrtc.poll_and_feed()?;
@@ -697,7 +706,8 @@ impl<S: CaptureSource> State<S> {
continue; continue;
} }
count += 1; count += 1;
if let Err(e) = wrtc.write_h264_frame(&data, self.webrtc_frames_sent, self.args.fps) { if let Err(e) = wrtc.write_h264_frame(&data, self.webrtc_frames_sent, self.args.fps)
{
tracing::debug!("WebRTC write frame error: {e}"); tracing::debug!("WebRTC write frame error: {e}");
} }
self.stats.record_send(0.0, None); self.stats.record_send(0.0, None);
@@ -709,10 +719,8 @@ impl<S: CaptureSource> State<S> {
} }
if self.args.stats && self.stats.should_snapshot() { if self.args.stats && self.stats.should_snapshot() {
self.stats.set_queue_depths( self.stats
0, .set_queue_depths(0, self.webrtc_rx.as_ref().map(|r| r.len()).unwrap_or(0));
self.webrtc_rx.as_ref().map(|r| r.len()).unwrap_or(0),
);
let snap = self.stats.snapshot_and_reset(); let snap = self.stats.snapshot_and_reset();
tracing::info!("stats: {snap}"); tracing::info!("stats: {snap}");
} }
@@ -752,9 +760,12 @@ impl<S: CaptureSource> State<S> {
.unwrap_or_else(|| 2 * (width as u64) * (height as u64) * (fps as u64) / 100); .unwrap_or_else(|| 2 * (width as u64) * (height as u64) * (fps as u64) / 100);
let enc = if let Some(ref tx) = self.webrtc_tx { let enc = if let Some(ref tx) = self.webrtc_tx {
let (enc_w, enc_h) = let (enc_w, enc_h) = transpose_if_transform_transposed(
transpose_if_transform_transposed(output_info.transform, width as i32, height as i32); output_info.transform,
let actual_gop_size = self.args.gop_size.unwrap_or((fps / 2).max(10)); width as i32,
height as i32,
);
let actual_gop_size = self.args.gop_size.unwrap_or((fps * 2).max(20));
match SwEncState::new_webrtc( match SwEncState::new_webrtc(
&drm_path, &drm_path,
width, width,
@@ -765,7 +776,10 @@ impl<S: CaptureSource> State<S> {
bitrate, bitrate,
actual_gop_size, actual_gop_size,
tx.clone(), tx.clone(),
self.webrtc_paused.as_ref().expect("webrtc_paused must exist when webrtc_tx exists").clone(), self.webrtc_paused
.as_ref()
.expect("webrtc_paused must exist when webrtc_tx exists")
.clone(),
) { ) {
Ok(enc) => StreamingEncoder::WebRtc(enc), Ok(enc) => StreamingEncoder::WebRtc(enc),
Err(e) => { Err(e) => {
@@ -775,7 +789,11 @@ impl<S: CaptureSource> State<S> {
} }
} }
} else { } else {
let output_path = self.args.output.as_deref().expect("output required for MP4 mode"); let output_path = self
.args
.output
.as_deref()
.expect("output required for MP4 mode");
match crate::avhw::create_encoder( match crate::avhw::create_encoder(
&drm_path, &drm_path,
Path::new(output_path), Path::new(output_path),
+237 -23
View File
@@ -8,7 +8,9 @@ use std::time::{Duration, Instant};
use anyhow::{bail, Result}; // 错误处理工具 use anyhow::{bail, Result}; // 错误处理工具
use crate::args::Args; // 命令行参数 use crate::args::Args; // 命令行参数
use crate::avhw::{self, CpuNv12Frame, SwEncEncode, SwEncImport, SwEncState}; // 软件编码器状态(VAAPI 导入 + H.264 编码) use crate::avhw::{
self, BitrateCommand, CpuNv12Frame, ResolutionChange, SwEncEncode, SwEncImport, SwEncState,
}; // 软件编码器状态(VAAPI 导入 + H.264 编码)
use crate::cap_portal::{CapPortal, PwCtrlEvent, PwDmaBufFrame}; // PipeWire 屏幕采集端点 use crate::cap_portal::{CapPortal, PwCtrlEvent, PwDmaBufFrame}; // PipeWire 屏幕采集端点
use crate::stats::{FrameTimings, PipelineStats}; // 管道统计(帧计时、每秒快照) use crate::stats::{FrameTimings, PipelineStats}; // 管道统计(帧计时、每秒快照)
use crate::webrtc::WebRtcState; // WebRTC 信令与媒体传输 use crate::webrtc::WebRtcState; // WebRTC 信令与媒体传输
@@ -147,7 +149,11 @@ impl StatePortal {
// 根据阻塞模式选择不同的帧接收策略 // 根据阻塞模式选择不同的帧接收策略
let frame = if block { let frame = if block {
// 阻塞模式:最多等待 2ms 接收帧 // 阻塞模式:最多等待 2ms 接收帧
match self.cap.frame_receiver().recv_timeout(std::time::Duration::from_millis(2)) { match self
.cap
.frame_receiver()
.recv_timeout(std::time::Duration::from_millis(2))
{
Ok(frame) => frame, Ok(frame) => frame,
Err(_) => { Err(_) => {
self.record_capture_timeout(); self.record_capture_timeout();
@@ -194,10 +200,10 @@ impl StatePortal {
let actual_bitrate = self.args.bitrate.unwrap_or_else(|| { let actual_bitrate = self.args.bitrate.unwrap_or_else(|| {
5 * (enc_width as u64) * (enc_height as u64) * (self.args.fps as u64) / 100 5 * (enc_width as u64) * (enc_height as u64) * (self.args.fps as u64) / 100
}); });
// GOP 大小:WebRTC 模式使用更小的 GOPfps/2,最低10),MP4 模式使用 fps // GOP 大小:WebRTC 模式使用较大的 GOPfps*2,最低20),MP4 模式使用 fps
let actual_gop_size = self.args.gop_size.unwrap_or_else(|| { let actual_gop_size = self.args.gop_size.unwrap_or_else(|| {
if self.webrtc.is_some() { if self.webrtc.is_some() {
(self.args.fps / 2).max(10) (self.args.fps * 2).max(20)
} else { } else {
self.args.fps self.args.fps
} }
@@ -207,17 +213,25 @@ impl StatePortal {
if self.webrtc.is_some() { if self.webrtc.is_some() {
let paused = self.webrtc_paused.as_ref() let paused = self.webrtc_paused.as_ref()
.ok_or_else(|| anyhow::anyhow!("internal invariant broken: webrtc_paused missing while WebRTC mode is active"))?; .ok_or_else(|| anyhow::anyhow!("internal invariant broken: webrtc_paused missing while WebRTC mode is active"))?;
let import = SwEncImport::new( let (resolution_tx, resolution_rx) =
crossbeam_channel::bounded::<BitrateCommand>(4);
let (encoder_resolution_tx, encoder_resolution_rx) =
crossbeam_channel::bounded::<ResolutionChange>(4);
let import = SwEncImport::new_with_resolution_control(
&drm_path, &drm_path,
frame.width, frame.width,
frame.height, frame.height,
enc_width, enc_width,
enc_height, enc_height,
self.args.fps, self.args.fps,
resolution_rx,
encoder_resolution_tx,
)?; )?;
let (webrtc_tx, webrtc_rx) = crossbeam_channel::bounded(2); let (webrtc_tx, webrtc_rx) = crossbeam_channel::bounded(2);
let (input_tx, input_rx) = crossbeam_channel::bounded::<CpuNv12Frame>(1); let (input_tx, input_rx) = crossbeam_channel::bounded::<CpuNv12Frame>(1);
let (timing_tx, timing_rx) = crossbeam_channel::bounded::<EncodeThreadTiming>(32); let (timing_tx, timing_rx) =
crossbeam_channel::bounded::<EncodeThreadTiming>(32);
let (bitrate_tx, bitrate_rx) = crossbeam_channel::bounded::<BitrateCommand>(4);
let encode = SwEncEncode::new_webrtc( let encode = SwEncEncode::new_webrtc(
enc_width, enc_width,
enc_height, enc_height,
@@ -226,24 +240,48 @@ impl StatePortal {
actual_gop_size, actual_gop_size,
webrtc_tx, webrtc_tx,
paused.clone(), paused.clone(),
bitrate_rx,
encoder_resolution_rx,
)?; )?;
let handle = std::thread::Builder::new() let handle = std::thread::Builder::new()
.name("wl-webrtc-encode".into()) .name("wl-webrtc-encode".into())
.spawn(move || encode_thread_loop(encode, input_rx, timing_tx))?; .spawn(move || encode_thread_loop(encode, input_rx, timing_tx))?;
self.enc_import = Some(import); self.enc_import = Some(import);
self.enc_thread = Some(EncodeThread { handle: Some(handle), input_tx, timing_rx }); self.enc_thread = Some(EncodeThread {
handle: Some(handle),
input_tx,
timing_rx,
});
let wrtc = self.webrtc.take() let wrtc = self.webrtc.take().ok_or_else(|| {
.ok_or_else(|| anyhow::anyhow!("internal: WebRtcState missing during init"))?; anyhow::anyhow!("internal: WebRtcState missing during init")
let paused = self.webrtc_paused.as_ref() })?;
let paused = self
.webrtc_paused
.as_ref()
.ok_or_else(|| anyhow::anyhow!("internal: webrtc_paused missing"))? .ok_or_else(|| anyhow::anyhow!("internal: webrtc_paused missing"))?
.clone(); .clone();
let fps = self.args.fps; let fps = self.args.fps;
let (sent_gap_tx, sent_gap_rx) = crossbeam_channel::bounded(64); let (sent_gap_tx, sent_gap_rx) = crossbeam_channel::bounded(64);
let webrtc_handle = std::thread::Builder::new() let webrtc_handle = std::thread::Builder::new()
.name("wl-webrtc-webrtc".into()) .name("wl-webrtc-webrtc".into())
.spawn(move || webrtc_thread_loop(wrtc, webrtc_rx, fps, paused, sent_gap_tx))?; .spawn(move || {
self.webrtc_thread = Some(WebrtcThread { handle: Some(webrtc_handle), sent_gap_rx }); webrtc_thread_loop(
wrtc,
webrtc_rx,
fps,
enc_width,
enc_height,
paused,
sent_gap_tx,
bitrate_tx,
resolution_tx,
)
})?;
self.webrtc_thread = Some(WebrtcThread {
handle: Some(webrtc_handle),
sent_gap_rx,
});
} else { } else {
// MP4 模式:编码输出写入文件 // MP4 模式:编码输出写入文件
let output_path = self.args.output.as_deref() let output_path = self.args.output.as_deref()
@@ -263,7 +301,9 @@ impl StatePortal {
}; };
self.stage = PortalStage::Streaming; // 切换到流式编码阶段 self.stage = PortalStage::Streaming; // 切换到流式编码阶段
self.start_time = Some(Instant::now()); self.start_time = Some(Instant::now());
tracing::info!("First frame processed, encoder initialized, transitioning to Streaming"); tracing::info!(
"First frame processed, encoder initialized, transitioning to Streaming"
);
drop(frame); // 首帧仅用于初始化,不参与编码 drop(frame); // 首帧仅用于初始化,不参与编码
} }
PortalStage::Streaming => { PortalStage::Streaming => {
@@ -295,7 +335,10 @@ impl StatePortal {
} }
let snap = self.stats.snapshot_and_reset(); let snap = self.stats.snapshot_and_reset();
if self.filler_frames_sent > 0 { if self.filler_frames_sent > 0 {
tracing::info!("stats: {snap} filler_frames_sent={}", self.filler_frames_sent); tracing::info!(
"stats: {snap} filler_frames_sent={}",
self.filler_frames_sent
);
} else { } else {
tracing::info!("stats: {snap}"); tracing::info!("stats: {snap}");
} }
@@ -321,9 +364,9 @@ impl StatePortal {
self.last_stall_log = Some(now); self.last_stall_log = Some(now);
tracing::warn!("compositor frame delivery stalled"); tracing::warn!("compositor frame delivery stalled");
} else { } else {
let should_log = self let should_log = self.last_stall_log.map_or(true, |last_log| {
.last_stall_log now.duration_since(last_log) >= Duration::from_secs(1)
.map_or(true, |last_log| now.duration_since(last_log) >= Duration::from_secs(1)); });
if should_log { if should_log {
self.last_stall_log = Some(now); self.last_stall_log = Some(now);
tracing::warn!("compositor frame delivery stalled"); tracing::warn!("compositor frame delivery stalled");
@@ -437,13 +480,11 @@ impl StatePortal {
} }
// 所有候选设备均失败,返回详细错误信息 // 所有候选设备均失败,返回详细错误信息
bail!( bail!(failures
failures
.into_iter() .into_iter()
.map(|(p, e)| format!("{} ({e})", p.display())) .map(|(p, e)| format!("{} ({e})", p.display()))
.collect::<Vec<_>>() .collect::<Vec<_>>()
.join(", ") .join(", "));
);
} }
/// 处理单帧 DMA-BUF 数据 /// 处理单帧 DMA-BUF 数据
@@ -513,8 +554,11 @@ impl StatePortal {
let import_us = t_import_start.elapsed().as_micros() as u64; let import_us = t_import_start.elapsed().as_micros() as u64;
self.stats.record_import(import_us); self.stats.record_import(import_us);
let enc_thread = self.enc_thread.as_ref() let enc_thread = self.enc_thread.as_ref().ok_or_else(|| {
.ok_or_else(|| anyhow::anyhow!("internal invariant broken: encode thread missing while async import is active"))?; anyhow::anyhow!(
"internal invariant broken: encode thread missing while async import is active"
)
})?;
let fillable_frame = CpuNv12Frame { let fillable_frame = CpuNv12Frame {
y_data: cpu_nv12.y_data.clone(), y_data: cpu_nv12.y_data.clone(),
uv_data: cpu_nv12.uv_data.clone(), uv_data: cpu_nv12.uv_data.clone(),
@@ -632,11 +676,20 @@ fn webrtc_thread_loop(
mut wrtc: WebRtcState, mut wrtc: WebRtcState,
webrtc_rx: crossbeam_channel::Receiver<Vec<u8>>, webrtc_rx: crossbeam_channel::Receiver<Vec<u8>>,
fps: u32, fps: u32,
enc_width: u32,
enc_height: u32,
paused: Arc<AtomicBool>, paused: Arc<AtomicBool>,
sent_gap_tx: crossbeam_channel::Sender<f64>, sent_gap_tx: crossbeam_channel::Sender<f64>,
bitrate_tx: crossbeam_channel::Sender<BitrateCommand>,
resolution_tx: crossbeam_channel::Sender<BitrateCommand>,
) { ) {
let mut frames_sent: u64 = 0; let mut frames_sent: u64 = 0;
let mut last_send: Option<std::time::Instant> = None; let mut last_send: Option<std::time::Instant> = None;
let mut last_sent_bitrate: Option<u64> = None;
let initial_tier = (enc_width, enc_height);
let mut current_tier = initial_tier;
let mut upscale_counter = 0u32;
let mut last_resolution_eval = Instant::now();
let timeout = Duration::from_millis(1); let timeout = Duration::from_millis(1);
loop { loop {
@@ -659,6 +712,52 @@ fn webrtc_thread_loop(
} }
paused.store(now_paused, Ordering::Relaxed); paused.store(now_paused, Ordering::Relaxed);
if let Some(bwe) = wrtc.get_bwe_estimate() {
let should_send = match last_sent_bitrate {
None => true,
Some(last) => {
let diff = if bwe > last { bwe - last } else { last - bwe };
diff * 10 > last
}
};
if should_send {
let _ = bitrate_tx.try_send(BitrateCommand::UpdateBitrate { target_bps: bwe });
last_sent_bitrate = Some(bwe);
}
if last_resolution_eval.elapsed() >= Duration::from_secs(1) {
last_resolution_eval = Instant::now();
let selected = select_resolution(current_tier.0, current_tier.1, bwe, fps);
if selected != current_tier {
current_tier = selected;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
} else if let Some(next_tier) = next_upscale_tier(current_tier, initial_tier) {
let needed = resolution_bitrate_bps(next_tier.0, next_tier.1, fps);
if bwe > needed.saturating_mul(120) / 100 {
upscale_counter = upscale_counter.saturating_add(1);
if upscale_counter >= 10 {
current_tier = next_tier;
upscale_counter = 0;
let _ = resolution_tx.try_send(BitrateCommand::UpdateResolution {
width: current_tier.0,
height: current_tier.1,
});
wrtc.set_need_keyframe();
}
} else {
upscale_counter = 0;
}
} else {
upscale_counter = 0;
}
}
}
if connected { if connected {
while let Ok(data) = webrtc_rx.try_recv() { while let Ok(data) = webrtc_rx.try_recv() {
if let Err(e) = wrtc.write_h264_frame(&data, frames_sent, fps) { if let Err(e) = wrtc.write_h264_frame(&data, frames_sent, fps) {
@@ -700,6 +799,43 @@ fn webrtc_thread_loop(
tracing::info!("WebRTC thread exiting"); tracing::info!("WebRTC thread exiting");
} }
const RESOLUTION_TIERS: &[(u32, u32)] = &[(2560, 1440), (1920, 1080), (1280, 720)];
fn resolution_bitrate_bps(width: u32, height: u32, fps: u32) -> u64 {
5 * u64::from(width) * u64::from(height) * u64::from(fps) / 100
}
/// Select resolution tier based on BWE estimate.
/// Returns (width, height) for the selected tier.
fn select_resolution(current_w: u32, current_h: u32, bwe_bps: u64, fps: u32) -> (u32, u32) {
let current = (current_w, current_h);
let current_bitrate = resolution_bitrate_bps(current_w, current_h, fps);
if bwe_bps >= current_bitrate.saturating_mul(60) / 100 {
return current;
}
let current_index = RESOLUTION_TIERS
.iter()
.position(|&tier| tier == current)
.unwrap_or_else(|| {
RESOLUTION_TIERS
.iter()
.position(|&(w, h)| w <= current_w && h <= current_h)
.unwrap_or(RESOLUTION_TIERS.len() - 1)
});
let next_index = (current_index + 1).min(RESOLUTION_TIERS.len() - 1);
RESOLUTION_TIERS[next_index]
}
fn next_upscale_tier(current: (u32, u32), ceiling: (u32, u32)) -> Option<(u32, u32)> {
let current_index = RESOLUTION_TIERS.iter().position(|&tier| tier == current)?;
if current_index == 0 {
return None;
}
let next = RESOLUTION_TIERS[current_index - 1];
(next.0 <= ceiling.0 && next.1 <= ceiling.1).then_some(next)
}
impl Drop for StatePortal { impl Drop for StatePortal {
// 析构时自动调用 shutdown,确保编码器被刷新、资源被释放 // 析构时自动调用 shutdown,确保编码器被刷新、资源被释放
fn drop(&mut self) { fn drop(&mut self) {
@@ -845,6 +981,40 @@ mod tests {
assert_eq!(result, None); assert_eq!(result, None);
} }
#[test]
fn select_resolution_downscales_one_tier_below_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 59 / 100, fps),
(1280, 720)
);
}
#[test]
fn select_resolution_keeps_tier_at_sixty_percent() {
let fps = 30;
let current = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, current * 60 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_never_goes_below_720p() {
assert_eq!(select_resolution(1280, 720, 1, 30), (1280, 720));
}
#[test]
fn next_upscale_tier_respects_initial_ceiling() {
assert_eq!(
next_upscale_tier((1280, 720), (1920, 1080)),
Some((1920, 1080))
);
assert_eq!(next_upscale_tier((1920, 1080), (1920, 1080)), None);
}
/// 测试:使用自定义偏移量和 stride 构建 DRM 描述符 /// 测试:使用自定义偏移量和 stride 构建 DRM 描述符
#[test] #[test]
fn build_drm_descriptor_custom_offset_and_stride() { fn build_drm_descriptor_custom_offset_and_stride() {
@@ -908,4 +1078,48 @@ mod tests {
)); ));
} }
// ── Task 7: Additional resolution tier edge cases ──
#[test]
fn select_resolution_keeps_720p_when_bwe_sufficient() {
let fps = 30;
let bitrate_720 = resolution_bitrate_bps(1280, 720, fps);
assert_eq!(
select_resolution(1280, 720, bitrate_720, fps),
(1280, 720)
);
}
#[test]
fn select_resolution_downscales_1440p_to_1080p() {
let fps = 30;
let bitrate_1440 = resolution_bitrate_bps(2560, 1440, fps);
assert_eq!(
select_resolution(2560, 1440, bitrate_1440 * 59 / 100, fps),
(1920, 1080)
);
}
#[test]
fn select_resolution_1080p_to_720p_at_very_low_bwe() {
let fps = 30;
let bitrate_1080 = resolution_bitrate_bps(1920, 1080, fps);
assert_eq!(
select_resolution(1920, 1080, bitrate_1080 / 10, fps),
(1280, 720)
);
}
#[test]
fn next_upscale_tier_from_720p_to_1080p() {
assert_eq!(
next_upscale_tier((1280, 720), (2560, 1440)),
Some((1920, 1080))
);
}
#[test]
fn next_upscale_tier_returns_none_at_highest() {
assert_eq!(next_upscale_tier((2560, 1440), (2560, 1440)), None);
}
} }
+8 -2
View File
@@ -461,14 +461,20 @@ mod tests {
// 100 values: 0.0 through 99.0 // 100 values: 0.0 through 99.0
let data: Vec<f64> = (0..100).map(|i| i as f64).collect(); let data: Vec<f64> = (0..100).map(|i| i as f64).collect();
let result = p95_f64(&data); let result = p95_f64(&data);
assert!((result - 95.0).abs() < 1.0, "p95 of 0..100 should be ~95, got {result}"); assert!(
(result - 95.0).abs() < 1.0,
"p95 of 0..100 should be ~95, got {result}"
);
} }
#[test] #[test]
fn p95_ms_microseconds() { fn p95_ms_microseconds() {
let data: Vec<u64> = (0..100).map(|i| i * 1000).collect(); // 0ms..99ms let data: Vec<u64> = (0..100).map(|i| i * 1000).collect(); // 0ms..99ms
let result = p95_ms(&data); let result = p95_ms(&data);
assert!((result - 95.0).abs() < 1.0, "p95_ms should be ~95ms, got {result}"); assert!(
(result - 95.0).abs() < 1.0,
"p95_ms should be ~95ms, got {result}"
);
} }
#[test] #[test]
+73 -17
View File
@@ -4,6 +4,7 @@ use std::net::{SocketAddr, TcpListener, UdpSocket};
use std::time::Instant; use std::time::Instant;
use anyhow::{bail, Result}; use anyhow::{bail, Result};
use str0m::bwe::{Bitrate, BweKind};
use str0m::change::SdpOffer; use str0m::change::SdpOffer;
use str0m::format::Codec; use str0m::format::Codec;
use str0m::media::{Frequency, MediaKind, MediaTime, Mid, Pt}; use str0m::media::{Frequency, MediaKind, MediaTime, Mid, Pt};
@@ -199,6 +200,7 @@ struct WebRtcInner {
video_pt: Option<Pt>, video_pt: Option<Pt>,
connected: bool, connected: bool,
need_keyframe: bool, need_keyframe: bool,
current_bwe_estimate: Option<Bitrate>,
rtp_clock: u32, rtp_clock: u32,
buf: Vec<u8>, buf: Vec<u8>,
} }
@@ -260,8 +262,7 @@ impl WebRtcState {
continue; continue;
} }
match WebRtcInner::new(self.fps) match WebRtcInner::new(self.fps).and_then(|mut new_inner| {
.and_then(|mut new_inner| {
let answer_json = new_inner.handle_sdp_offer(body.as_bytes())?; let answer_json = new_inner.handle_sdp_offer(body.as_bytes())?;
Ok((new_inner, answer_json)) Ok((new_inner, answer_json))
}) { }) {
@@ -285,7 +286,8 @@ impl WebRtcState {
} }
Err(e) => { Err(e) => {
tracing::error!("SDP offer handling failed: {e}"); tracing::error!("SDP offer handling failed: {e}");
let resp = "HTTP/1.1 500 Internal Server Error\r\nConnection: close\r\n\r\n"; let resp =
"HTTP/1.1 500 Internal Server Error\r\nConnection: close\r\n\r\n";
if let Err(e) = stream.write_all(resp.as_bytes()) { if let Err(e) = stream.write_all(resp.as_bytes()) {
tracing::debug!("HTTP write error: {e}"); tracing::debug!("HTTP write error: {e}");
} }
@@ -340,12 +342,27 @@ impl WebRtcState {
pub fn is_connected(&self) -> bool { pub fn is_connected(&self) -> bool {
self.inner.as_ref().is_some_and(WebRtcInner::is_connected) self.inner.as_ref().is_some_and(WebRtcInner::is_connected)
} }
/// Returns the latest bandwidth estimation estimate in bits per second, if available.
pub fn get_bwe_estimate(&self) -> Option<u64> {
self.inner
.as_ref()
.and_then(|inner| inner.current_bwe_estimate.map(|b| b.as_u64()))
}
pub fn set_need_keyframe(&mut self) {
if let Some(inner) = self.inner.as_mut() {
inner.need_keyframe = true;
}
}
} }
impl WebRtcInner { impl WebRtcInner {
fn new(fps: u32) -> Result<Self> { fn new(fps: u32) -> Result<Self> {
let _ = fps; let _ = fps;
let mut rtc = RtcConfig::new().build(Instant::now()); let mut rtc = RtcConfig::new()
.enable_bwe(Some(Bitrate::mbps(5)))
.build(Instant::now());
let socket = UdpSocket::bind("0.0.0.0:0")?; let socket = UdpSocket::bind("0.0.0.0:0")?;
socket.set_nonblocking(true)?; socket.set_nonblocking(true)?;
@@ -368,10 +385,14 @@ impl WebRtcInner {
std::mem::size_of::<libc::c_int>() as libc::socklen_t, std::mem::size_of::<libc::c_int>() as libc::socklen_t,
); );
if ret < 0 { if ret < 0 {
tracing::warn!("setsockopt SO_SNDBUF failed (errno {})", std::io::Error::last_os_error()); tracing::warn!(
"setsockopt SO_SNDBUF failed (errno {})",
std::io::Error::last_os_error()
);
} }
let mut actual: libc::c_int = 0; let mut actual: libc::c_int = 0;
let mut actual_len: libc::socklen_t = std::mem::size_of::<libc::c_int>() as libc::socklen_t; let mut actual_len: libc::socklen_t =
std::mem::size_of::<libc::c_int>() as libc::socklen_t;
let gret = libc::getsockopt( let gret = libc::getsockopt(
fd, fd,
libc::SOL_SOCKET, libc::SOL_SOCKET,
@@ -408,14 +429,15 @@ impl WebRtcInner {
video_pt: None, video_pt: None,
connected: false, connected: false,
need_keyframe: false, need_keyframe: false,
current_bwe_estimate: None,
rtp_clock: 0, rtp_clock: 0,
buf: vec![0u8; 65535], buf: vec![0u8; 65535],
}) })
} }
fn handle_sdp_offer(&mut self, body: &[u8]) -> Result<String> { fn handle_sdp_offer(&mut self, body: &[u8]) -> Result<String> {
let offer: SdpOffer = serde_json::from_slice(body) let offer: SdpOffer =
.map_err(|e| anyhow::anyhow!("parse SDP offer: {e}"))?; serde_json::from_slice(body).map_err(|e| anyhow::anyhow!("parse SDP offer: {e}"))?;
let answer = self let answer = self
.rtc .rtc
@@ -492,9 +514,7 @@ impl WebRtcInner {
tracing::info!("Media added: mid={} kind={:?}", ma.mid, ma.kind); tracing::info!("Media added: mid={} kind={:?}", ma.mid, ma.kind);
if ma.kind == MediaKind::Video { if ma.kind == MediaKind::Video {
if let Some(media) = self.rtc.media(ma.mid) { if let Some(media) = self.rtc.media(ma.mid) {
if media.direction().is_sending() if media.direction().is_sending() && self.video_mid.is_none() {
&& self.video_mid.is_none()
{
self.video_mid = Some(ma.mid); self.video_mid = Some(ma.mid);
tracing::info!("Captured video mid: {}", ma.mid); tracing::info!("Captured video mid: {}", ma.mid);
self.discover_video_params(); self.discover_video_params();
@@ -502,6 +522,22 @@ impl WebRtcInner {
} }
} }
} }
Event::KeyframeRequest(_) => {
tracing::info!("received keyframe request from viewer");
self.need_keyframe = true;
}
Event::EgressBitrateEstimate(est) => {
let bitrate = match est {
BweKind::Twcc(b) => *b,
BweKind::Remb(_, b) => *b,
_ => {
tracing::debug!("BWE estimate: unrecognized kind, skipping");
continue;
}
};
tracing::info!("BWE estimate: {bitrate}");
self.current_bwe_estimate = Some(bitrate);
}
_ => { _ => {
tracing::debug!("WebRTC event: {:?}", e); tracing::debug!("WebRTC event: {:?}", e);
} }
@@ -538,9 +574,9 @@ impl WebRtcInner {
.map_err(|e| anyhow::anyhow!("receive contents: {e}"))?, .map_err(|e| anyhow::anyhow!("receive contents: {e}"))?,
}, },
); );
self.rtc self.rtc.handle_input(input).map_err(|e| {
.handle_input(input) anyhow::anyhow!("handle_input({n} bytes from {source}): {e}")
.map_err(|e| anyhow::anyhow!("handle_input({n} bytes from {source}): {e}"))?; })?;
} }
Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => break, Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => break,
Err(ref e) if e.kind() == std::io::ErrorKind::Interrupted => continue, Err(ref e) if e.kind() == std::io::ErrorKind::Interrupted => continue,
@@ -636,9 +672,7 @@ fn extract_body(req: &str) -> &str {
} }
fn local_ip() -> Option<String> { fn local_ip() -> Option<String> {
std::net::UdpSocket::bind("0.0.0.0:0") std::net::UdpSocket::bind("0.0.0.0:0").ok().and_then(|s| {
.ok()
.and_then(|s| {
s.connect("1.1.1.1:80").ok()?; s.connect("1.1.1.1:80").ok()?;
let addr = s.local_addr().ok()?; let addr = s.local_addr().ok()?;
drop(s); drop(s);
@@ -746,4 +780,26 @@ mod tests {
fn all_zeros() { fn all_zeros() {
assert!(!is_idr_nalu(&[0, 0, 0, 0, 0, 0, 0, 0])); assert!(!is_idr_nalu(&[0, 0, 0, 0, 0, 0, 0, 0]));
} }
// ── Task 5: BWE estimate handling ──
#[test]
fn bitrate_mbps_conversion() {
let b = Bitrate::mbps(5);
assert!(b.as_u64() > 0, "5 Mbps should be > 0 bps");
assert_eq!(b.as_u64(), 5_000_000, "5 Mbps should be 5,000,000 bps");
}
#[test]
fn bitrate_bps_conversion() {
let b = Bitrate::bps(1_234_567);
assert_eq!(b.as_u64(), 1_234_567);
}
#[test]
fn default_bwe_is_5mbps() {
let default = Bitrate::mbps(5);
let bps = default.as_u64();
assert_eq!(bps, 5_000_000);
}
} }