diff --git a/AGENTS.md b/AGENTS.md index 65e055c..9a41b4d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -282,12 +282,16 @@ started a second full host server instead, so check that a subcommand exists in The FFmpeg process pipeline uploads NV12 frames to a VAAPI frame pool. With `--features inproc-av` on X11, VAAPI H.264/HEVC Main runs in process: a DRI3 DMA-BUF of the root (XShm upload without DRI3) goes through VPP to NV12 and VCN, with SPS/PPS(/VPS) and slice headers packed by the host as radeonsi (Mesa 26) needs; parameters follow -FFmpeg's hevc_vaapi/h264_vaapi on that driver. Accepted on AMD HawkPoint1 (Mesa 26.2.3) with the official -Mac viewer (presets, fps, scaling); Intel iHD is not verified. +FFmpeg's hevc_vaapi/h264_vaapi on that driver. Accepted on AMD HawkPoint1 (Mesa 26.2.3) and Cezanne +(Mesa 26.2.5; XShm-upload and DRI3 zero-copy probes plus a full HEVC Mac-viewer session) with the official +Mac viewer (presets, fps, scaling); Intel iHD is not verified. Measured on Cezanne + RTX 3050 Ti +(1920x1200, desktop-video-probe, 300 frames): in-process VAAPI ~1.5 ms CPU/frame vs in-process NVENC +~7.1 ms vs libx264 process pipeline ~19 ms. VBR uses the viewer bitrate preset and CQ-derived minimum QP, no B-frames, async depth 1, AUD and the existing GOP/VFR timing. Unsupported device/driver/initialization falls back to libx264 with a warning. Accepted on Intel UHD 770 (iHD) with the default FFmpeg process pipeline and the official Mac -viewer (presets, fps, scaling, reconnect; no fallback, ~40% less CPU than libx264). AMD is not hardware-verified. +viewer (presets, fps, scaling, reconnect; no fallback, ~40% less CPU than libx264). The process pipeline on +AMD is not separately verified (AMD acceptance is the in-process engine above). ### Virtual audio / 虚拟声卡 diff --git a/README.en.md b/README.en.md index 876a251..96f93af 100644 --- a/README.en.md +++ b/README.en.md @@ -172,6 +172,8 @@ sudo apt install build-essential pkg-config \ sudo apt install libwayland-dev libxkbcommon-dev wl-clipboard # optional: GNOME/KDE portal backend sudo apt install libglib2.0-dev libpipewire-0.3-dev libei-dev +# optional: headers for the inproc-av VAAPI/DRI3 engines (libva itself is loaded at runtime) +sudo apt install libva-dev libgbm-dev libx11-xcb-dev libxcb-dri3-dev libxcb-damage0-dev ``` - A Rust toolchain is required (). @@ -188,7 +190,7 @@ cargo build --release --features wayland,portal # + GNOME/KDE on Wayland ``` The viewer's `viewer-ffmpeg` feature supports **FFmpeg 6.x only** (`pkg-config --modversion libavcodec` = 60.x); FFmpeg 7+ fails inside `ffmpeg-sys-next` with `avfft.h file not found` (#63). -The host's optional `--features inproc-av` encodes with NVENC inside the process on X11. It does not link FFmpeg development libraries; it loads the NVIDIA driver libraries (`libcuda.so.1`, `libnvidia-encode.so.1`) at runtime and needs a driver with NVENC API 12.1 (530 or newer). When the capture and encoder sizes match, NVENC reads the XShm buffer directly, saving one host-to-device copy. CPU and VAAPI encoding and `UU_CAPTURE=grab` keep the FFmpeg process; if the in-process NVENC session fails it falls back to FFmpeg's NVENC process pipeline, then to the CPU. The default build and the release packages need neither feature. +The host's optional `--features inproc-av` encodes with hardware inside the process on X11. It does not link FFmpeg development libraries; it loads the NVIDIA driver libraries (`libcuda.so.1`, `libnvidia-encode.so.1`) at runtime for NVENC (needs a driver with NVENC API 12.1, 530 or newer) and `libva.so.2`/`libva-drm.so.2` for VAAPI. When the capture and encoder sizes match, NVENC reads the XShm buffer directly, saving one host-to-device copy; VAAPI runs in process under the same feature (DRI3 DMA-BUF zero copy, or XShm upload without DRI3, then VPP to NV12 and VCN; H.264/HEVC). CPU encoding and `UU_CAPTURE=grab` keep the FFmpeg process; if the in-process hardware session fails it falls back to the matching FFmpeg process pipeline, then to the CPU. The default build and the release packages need neither feature. ### 3. Log in @@ -427,11 +429,21 @@ sends about 12 kbit/s. `UU_VIDEO_CODEC=h265` selects H.265 explicitly for A/B di `UU_VIDEO_ENCODER=vaapi` selects Intel/AMD H.264 VAAPI; `UU_VAAPI_DEVICE` defaults to `/dev/dri/renderD128` (render-node permissions required). No automatic VAAPI selection. -The FFmpeg process uploads NV12 frames to a VAAPI frame pool. +The FFmpeg process uploads NV12 frames to a VAAPI frame pool; a self-built `--features inproc-av` +binary instead runs VAAPI in process on X11 (a DRI3 DMA-BUF of the root window zero-copy, or XShm +upload without DRI3, then VPP to NV12 and VCN; H.264/HEVC). VBR uses the viewer bitrate preset and CQ-derived minimum QP, no B-frames, async depth 1, AUD and the existing GOP/VFR timing. Unsupported device/driver/initialization falls back to libx264 with a warning. Accepted on Intel UHD 770 (iHD) with the default FFmpeg process pipeline and the official Mac -viewer (presets, fps, scaling, reconnect; no fallback, ~40% less CPU than libx264). AMD is not hardware-verified. +viewer (presets, fps, scaling, reconnect; no fallback, ~40% less CPU than libx264 — a figure from the +FFmpeg-process-pipeline era). AMD iGPUs are verified on +the **in-process engine** (`--features inproc-av`): end-to-end sessions with the official Mac viewer passed +acceptance on HawkPoint (Mesa 26.2.3) and Cezanne (Mesa 26.2.5) — HEVC, presets, fps, scaling; on Cezanne, +H.264 and both capture paths were additionally probe-verified: the XShm upload (decodable bitstream, correct +picture) and the DRI3 zero-copy import. Measured on one machine (1920x1200@60, Cezanne iGPU + RTX 3050 Ti): +in-process VAAPI costs about **1/12** of the per-frame encode CPU of libx264 (FFmpeg pipeline, superfast) +and about **1/5** of in-process NVENC. The FFmpeg process pipeline (what the release packages use) is not +separately verified on AMD. The default VAAPI VBV buffer (100-500 kbit) squeezes keyframes hard at high resolution: on a 2560x1440 H.264 VAAPI stream capped at 30 Mbit/s the first IDR measured 59,910 bytes and the picture blurred at diff --git a/README.md b/README.md index f4a3fad..48ebf9c 100644 --- a/README.md +++ b/README.md @@ -157,6 +157,8 @@ sudo apt install build-essential pkg-config \ sudo apt install libwayland-dev libxkbcommon-dev wl-clipboard # 可选:GNOME/KDE portal 后端 sudo apt install libglib2.0-dev libpipewire-0.3-dev libei-dev +# 可选:inproc-av 的 VAAPI/DRI3 引擎编译所需(libva 本体运行时加载) +sudo apt install libva-dev libgbm-dev libx11-xcb-dev libxcb-dri3-dev libxcb-damage0-dev ``` - 需要 Rust 工具链()。 @@ -172,7 +174,7 @@ cargo build --release --features wayland,portal # + GNOME/KDE Wayland ``` 控制端的 `viewer-ffmpeg` 只支持 **FFmpeg 6.x**(`pkg-config --modversion libavcodec` 为 60.x);FFmpeg 7 及以上会在依赖 `ffmpeg-sys-next` 里报 `avfft.h file not found`(#63)。 -服务端可选的 `--features inproc-av` 是 X11 下的进程内 NVENC 编码:不链接 FFmpeg 开发库,运行时加载 NVIDIA 驱动库(`libcuda.so.1`、`libnvidia-encode.so.1`),需要支持 NVENC API 12.1 的驱动(530 及以上)。截屏与编码尺寸一致时 NVENC 直接读取 XShm 缓冲,省去一次主机到显存的拷贝。CPU、VAAPI 编码或 `UU_CAPTURE=grab` 仍走外部 FFmpeg 进程;进程内 NVENC 启动失败时先回退到 FFmpeg 的 NVENC 进程管线,再回退到 CPU。默认构建和预编译包都不需要该特性。 +服务端可选的 `--features inproc-av` 是 X11 下的进程内硬件编码:不链接 FFmpeg 开发库,NVENC 运行时加载 NVIDIA 驱动库(`libcuda.so.1`、`libnvidia-encode.so.1`,需要支持 NVENC API 12.1 的驱动,530 及以上),VAAPI 运行时加载 `libva.so.2`/`libva-drm.so.2`。截屏与编码尺寸一致时 NVENC 直接读取 XShm 缓冲,省去一次主机到显存的拷贝;VAAPI 在同一特性下也进程内运行(DRI3 DMA-BUF 零拷贝,无 DRI3 时 XShm 上传,VPP 转 NV12 后送 VCN,支持 H.264/H.265)。CPU 编码或 `UU_CAPTURE=grab` 仍走外部 FFmpeg 进程;进程内硬件编码启动失败时先回退到对应的 FFmpeg 进程管线,再回退到 CPU。默认构建和预编译包都不需要该特性。 ### 3. 登录 @@ -385,7 +387,7 @@ HEVC 的访问单元按 RFC 7798 的图片边界切分,不依赖 AUD:`hevc_v `/dev/dri/renderD128`,运行用户需要 render 节点权限。不会自动选择 VAAPI;设备或驱动初始化 失败会明确警告并回退 libx264。外部 FFmpeg 先转 NV12 再上传硬件帧。 VBR 使用控制端码率预设及 CQ 对应的最小 QP,无 B 帧、async depth=1,并保留现有 GOP/VFR 规则。 -Intel 核显(UHD 770,iHD 驱动)已实机验收:默认的外部 FFmpeg 管线用官方 Mac 控制端测试了各档画质、帧率、缩放和重连,全程硬件编码、无回退,CPU 占用比 libx264 低约 40%。**AMD 显卡的 VAAPI 尚未实机验证。** +Intel 核显(UHD 770,iHD 驱动)已实机验收:默认的外部 FFmpeg 管线用官方 Mac 控制端测试了各档画质、帧率、缩放和重连,全程硬件编码、无回退,CPU 占用比 libx264 低约 40%(该数字为 FFmpeg 进程管线时代的测量)。AMD 核显已在**进程内原生引擎**上实机验收(`--features inproc-av`):官方 Mac 客户端的端到端会话在 HawkPoint(Mesa 26.2.3)与 Cezanne(Mesa 26.2.5)上通过验收(HEVC,含画质档、帧率、缩放);Cezanne 上另以探测验证了 H.264 与两条采集路径——XShm 上传(码流可解码、画面正确)与 DRI3 DMA-BUF 零拷贝导入。同机实测(1920×1200@60,Cezanne iGPU + RTX 3050 Ti 双显卡):进程内 VAAPI 每帧编码 CPU 约为 libx264(FFmpeg 管线 superfast)的 **1/12**、进程内 NVENC 的 **1/5**;外部 FFmpeg 进程管线(预编译包使用)在 AMD 上未单独验证。 高分辨率下 VAAPI 的默认 VBV 缓冲(100–500 kbit)会把关键帧压得过狠:2560×1440 H.264 VAAPI、30M 上限、静止画面上实测首个 IDR 只有 59,910 字节,画面每隔一个 GOP 糊一下再慢慢恢复。设