inproc-av: native in-process NVENC reading XShm directly (#74, #75)

Replaces the FFmpeg-based in-process encoder with a native NVENC engine (vendored nv-codec-headers n12.1.14.0, libcuda/libnvidia-encode dlopen'ed at runtime, primary context reuse). Fallback: native NVENC -> FFmpeg NVENC process -> libx264. inproc-av no longer links FFmpeg; VAAPI/CPU always use the process pipeline. Default builds are unchanged.

Co-authored-by: ferstar <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
authored and GitHub committed 2026-10-21 07:26:45 -07:00
1 parent 1fa346fd1f
commit 76941459cc
17 files changed
+6205 -454

No files matched your search

+2 -3
View File
@@ -94,12 +94,11 @@ jobs:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- name: Install build dependencies (X11 + FFmpeg 6 for the inproc-av feature)
- name: Install build dependencies (X11; inproc-av vendors the NVENC headers)
run: |
sudo apt-get update
sudo apt-get install -y build-essential pkg-config \
libx11-dev libxext-dev libxtst-dev libxfixes-dev libgtk-3-dev \
libavcodec-dev libavformat-dev libavdevice-dev libswscale-dev libclang-dev
libx11-dev libxext-dev libxtst-dev libxfixes-dev libgtk-3-dev
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- run: cargo test --locked --features inproc-av
+10 -9
View File
@@ -38,9 +38,10 @@ Dependencies (Ubuntu/Debian):
```sh
sudo apt install build-essential pkg-config \
libx11-dev libxext-dev libxtst-dev libxfixes-dev libavcodec-dev libavformat-dev libavdevice-dev libswscale-dev libclang-dev libgtk-3-dev ffmpeg
# --features inproc-av: libavcodec-dev libavformat-dev libavdevice-dev libswscale-dev libclang-dev
# (FFmpeg 6.x only; in-process capture/encoding, Ubuntu 24.04 ships it. FFmpeg >= 7
# fails in ffmpeg-sys-next with "avfft.h file not found", before our build.rs runs)
# --features inproc-av: nothing extra (X11 in-process capture + native NVENC; vendored NVENC headers,
# libcuda/libnvidia-encode are dlopen'ed at runtime; NVENC API 12.1, driver >= 530)
# --features viewer-ffmpeg: libavcodec-dev libavformat-dev libavdevice-dev libswscale-dev libclang-dev
# (FFmpeg 6.x only. FFmpeg >= 7 fails in ffmpeg-sys-next with "avfft.h file not found")
# --features wayland: libwayland-dev libxkbcommon-dev wl-clipboard
# --features wayland,portal: + libglib2.0-dev libpipewire-0.3-dev libei-dev (libei >= 1.1)
```
@@ -183,7 +184,7 @@ started a second full host server instead, so check that a subcommand exists in
5 Auto, 6 Custom. Resolution: 16/17 pixel size or 5/6, 10 resolution_type (3 = follow remote).
Field 7 `chorma_format` (ChromaFormat 1 = 4:2:0, 3 = 4:4:4) is sent when 真彩 is toggled; ff_capture_setting versions
are cumulative (3 CHROMA_SAMPLING, 4 CUSTOM_BITRATE, 5 DPI_SCALE, 6 HDR). 4:4:4 = NVENC `high444p` / x264 `high444`
with `yuv444p`; VAAPI and inproc stay 4:2:0.
with `yuv444p`; VAAPI and the in-process NVENC engine stay 4:2:0.
Reply `RpcResponse{2 CaptureSettingResponse{}}` (no errors = applied) and send Message 25 CaptureSettingSync
(field 3 = the chroma actually encoded).
- ConnectOptions field 4 `decoder_cap_list` = DecoderCap {1 fps, 2 codec_type (1 H264, 2 H265), 3 width, 4 height,
@@ -275,12 +276,12 @@ started a second full host server instead, so check that a subcommand exists in
`UU_VIDEO_ENCODER=vaapi` selects Intel/AMD H.264 VAAPI; `UU_VAAPI_DEVICE` defaults to
`/dev/dri/renderD128` (render-node permissions required). No automatic VAAPI selection.
Both FFmpeg process and `--features inproc-av` upload NV12 frames to a VAAPI frame pool.
The FFmpeg process pipeline uploads NV12 frames to a VAAPI frame pool (`--features inproc-av` is NVENC only;
VAAPI and CPU encoders always use the process pipeline).
VBR uses the viewer bitrate preset and CQ-derived minimum QP, no B-frames, async depth 1,
AUD and the existing GOP/VFR timing. Unsupported device/driver/initialization falls back to libx264
with a warning. Accepted on Intel UHD 770 (iHD) with the default FFmpeg process pipeline and the official Mac
viewer (presets, fps, scaling, reconnect; no fallback, ~40% less CPU than libx264). AMD and the `inproc-av`
VAAPI path are not hardware-verified.
viewer (presets, fps, scaling, reconnect; no fallback, ~40% less CPU than libx264). AMD is not hardware-verified.
### Virtual audio / 虚拟声卡
@@ -321,8 +322,8 @@ This permission can inject into the whole seat, including lock/greeter screens.
Optional VAAPI: `intel-media-driver` (modern Intel) or `libva-mesa-driver` (AMD), `libva-utils`
for `vainfo`. Optional KMS: `libdrm`. Virtual Xfce: `xorg-server-xvfb xorg-xauth xorg-xdpyinfo
xfce4-session xfce4-panel xfdesktop xfwm4 gnome-terminal`.
Rolling Arch FFmpeg may be newer than supported `inproc-av` (FFmpeg 6.x); use the default process
pipeline or build against a separate FFmpeg 6 prefix. Do not downgrade the system FFmpeg.
Rolling Arch FFmpeg may be newer than supported `viewer-ffmpeg` (FFmpeg 6.x); build it against a separate
FFmpeg 6 prefix. Do not downgrade the system FFmpeg. The host's `inproc-av` does not link FFmpeg.
User-bus headless setup and the official Mac checklist: [virtual desktop acceptance](contrib/virtual-desktop/ACCEPTANCE.md).
### Experimental KMS / 实验后端
+8 -3
View File
@@ -30,9 +30,9 @@ kms = []
portal = []
# Build the wayland/portal engines as runtime-loaded libuu-wayland.so / libuu-portal.so (release packages).
engine-plugins = []
# In-process capture and encoding through FFmpeg 6 libraries (libavcodec/libavdevice dev packages).
# Off by default so release binaries do not link the build machine's libav.
inproc-av = ["dep:ffmpeg-next", "dep:x11rb"]
# In-process X11 capture and NVENC encoding: vendored NVENC headers, NVIDIA driver libraries loaded at runtime.
# Other encoders keep the FFmpeg process pipeline. Off by default.
inproc-av = ["dep:x11rb"]
[dependencies]
tokio = { version = "1", features = ["full"] }
@@ -67,3 +67,8 @@ cc = "1.0"
webrtc-dtls = { path = "vendor/webrtc-dtls" }
# Larger data-channel receive buffer (see vendor/webrtc/src/data_channel/mod.rs).
webrtc = { path = "vendor/webrtc" }
[[bench]]
name = "nvenc_pipeline_bench"
harness = false
required-features = ["inproc-av"]
+5 -9
View File
@@ -187,10 +187,8 @@ cargo build --release --features wayland # + Hyprland
cargo build --release --features wayland,portal # + GNOME/KDE on Wayland
```
The optional `--features inproc-av` (in-process encoding) and the viewer's `viewer-ffmpeg` support **FFmpeg 6.x only**
(`pkg-config --modversion libavcodec` = 60.x). FFmpeg 7+ fails inside the `ffmpeg-sys-next` dependency with
`avfft.h file not found`; its build script runs before this project's, so no clearer check is possible (#63). The default
build and the release packages need neither feature.
The viewer's `viewer-ffmpeg` feature supports **FFmpeg 6.x only** (`pkg-config --modversion libavcodec` = 60.x); FFmpeg 7+ fails inside `ffmpeg-sys-next` with `avfft.h file not found` (#63).
The host's optional `--features inproc-av` encodes with NVENC inside the process on X11. It does not link FFmpeg development libraries; it loads the NVIDIA driver libraries (`libcuda.so.1`, `libnvidia-encode.so.1`) at runtime and needs a driver with NVENC API 12.1 (530 or newer). When the capture and encoder sizes match, NVENC reads the XShm buffer directly, saving one host-to-device copy. CPU and VAAPI encoding and `UU_CAPTURE=grab` keep the FFmpeg process; if the in-process NVENC session fails it falls back to FFmpeg's NVENC process pipeline, then to the CPU. The default build and the release packages need neither feature.
### 3. Log in
@@ -429,12 +427,11 @@ sends about 12 kbit/s. `UU_VIDEO_CODEC=h265` selects H.265 explicitly for A/B di
`UU_VIDEO_ENCODER=vaapi` selects Intel/AMD H.264 VAAPI; `UU_VAAPI_DEVICE` defaults to
`/dev/dri/renderD128` (render-node permissions required). No automatic VAAPI selection.
Both FFmpeg process and `--features inproc-av` upload NV12 frames to a VAAPI frame pool.
The FFmpeg process uploads NV12 frames to a VAAPI frame pool.
VBR uses the viewer bitrate preset and CQ-derived minimum QP, no B-frames, async depth 1,
AUD and the existing GOP/VFR timing. Unsupported device/driver/initialization falls back to libx264
with a warning. Accepted on Intel UHD 770 (iHD) with the default FFmpeg process pipeline and the official Mac
viewer (presets, fps, scaling, reconnect; no fallback, ~40% less CPU than libx264). AMD and the `inproc-av`
VAAPI path are not hardware-verified.
viewer (presets, fps, scaling, reconnect; no fallback, ~40% less CPU than libx264). AMD is not hardware-verified.
The default VAAPI VBV buffer (100-500 kbit) squeezes keyframes hard at high resolution: on a 2560x1440
H.264 VAAPI stream capped at 30 Mbit/s the first IDR measured 59,910 bytes and the picture blurred at
@@ -490,8 +487,7 @@ Keep the unit name `uuremote-native.service` (not a template `[email protected]
Optional VAAPI: `intel-media-driver` (modern Intel) or `libva-mesa-driver` (AMD), `libva-utils`
for `vainfo`. Optional KMS: `libdrm`. Virtual Xfce: `xorg-server-xvfb xorg-xauth xorg-xdpyinfo
xfce4-session xfce4-panel xfdesktop xfwm4 gnome-terminal`.
Rolling Arch FFmpeg may be newer than supported `inproc-av` (FFmpeg 6.x); use the default process
pipeline or build against a separate FFmpeg 6 prefix. Do not downgrade the system FFmpeg.
If building `viewer-ffmpeg`, rolling Arch FFmpeg may be newer than supported 6.x; use a separate FFmpeg 6 prefix rather than downgrading system FFmpeg (`inproc-av` native NVENC on the host does not depend on FFmpeg).
User-bus headless setup and the official Mac checklist: [virtual desktop acceptance](contrib/virtual-desktop/ACCEPTANCE.md).
### Experimental KMS / 实验后端
+5 -7
View File
@@ -171,9 +171,8 @@ cargo build --release --features wayland # + Hyprland
cargo build --release --features wayland,portal # + GNOME/KDE Wayland
```
可选的 `--features inproc-av`(进程内编码)和控制端的 `viewer-ffmpeg` 只支持 **FFmpeg 6.x**(`pkg-config --modversion libavcodec` 为 60.x)。
FFmpeg 7 及以上会在依赖 `ffmpeg-sys-next` 里报 `avfft.h file not found`——它的构建脚本先于本项目运行,所以无法提前给出更清楚的提示(#63)。
默认构建和预编译包都不需要这两个特性。
控制端的 `viewer-ffmpeg` 只支持 **FFmpeg 6.x**(`pkg-config --modversion libavcodec` 为 60.x);FFmpeg 7 及以上会在依赖 `ffmpeg-sys-next` 里报 `avfft.h file not found`(#63)。
服务端可选的 `--features inproc-av` 是 X11 下的进程内 NVENC 编码:不链接 FFmpeg 开发库,运行时加载 NVIDIA 驱动库(`libcuda.so.1`、`libnvidia-encode.so.1`),需要支持 NVENC API 12.1 的驱动(530 及以上)。截屏与编码尺寸一致时 NVENC 直接读取 XShm 缓冲,省去一次主机到显存的拷贝。CPU、VAAPI 编码或 `UU_CAPTURE=grab` 仍走外部 FFmpeg 进程;进程内 NVENC 启动失败时先回退到 FFmpeg 的 NVENC 进程管线,再回退到 CPU。默认构建和预编译包都不需要该特性。
### 3. 登录
@@ -384,9 +383,9 @@ HEVC 的访问单元按 RFC 7798 的图片边界切分,不依赖 AUD:`hevc_v
设置 `UU_VIDEO_ENCODER=vaapi` 使用 Intel/AMD H.264 VAAPI;`UU_VAAPI_DEVICE` 默认
`/dev/dri/renderD128`,运行用户需要 render 节点权限。不会自动选择 VAAPI;设备或驱动初始化
失败会明确警告并回退 libx264。外部 FFmpeg 和 `--features inproc-av` 均先转 NV12 再上传硬件帧。
失败会明确警告并回退 libx264。外部 FFmpeg 先转 NV12 再上传硬件帧。
VBR 使用控制端码率预设及 CQ 对应的最小 QP,无 B 帧、async depth=1,并保留现有 GOP/VFR 规则。
Intel 核显(UHD 770,iHD 驱动)已实机验收:默认的外部 FFmpeg 管线用官方 Mac 控制端测试了各档画质、帧率、缩放和重连,全程硬件编码、无回退,CPU 占用比 libx264 低约 40%。**AMD 显卡和 `--features inproc-av` 管线的 VAAPI 尚未实机验证。**
Intel 核显(UHD 770,iHD 驱动)已实机验收:默认的外部 FFmpeg 管线用官方 Mac 控制端测试了各档画质、帧率、缩放和重连,全程硬件编码、无回退,CPU 占用比 libx264 低约 40%。**AMD 显卡的 VAAPI 尚未实机验证。**
高分辨率下 VAAPI 的默认 VBV 缓冲(100–500 kbit)会把关键帧压得过狠:2560×1440 H.264 VAAPI、30M
上限、静止画面上实测首个 IDR 只有 59,910 字节,画面每隔一个 GOP 糊一下再慢慢恢复。设
@@ -451,8 +450,7 @@ WantedBy=graphical-session.target
VAAPI 可选包:Intel 新显卡用 `intel-media-driver`,AMD 用 `libva-mesa-driver`,`libva-utils` 提供
`vainfo`。KMS 依赖 `libdrm`。虚拟 Xfce 依赖 `xorg-server-xvfb xorg-xauth xorg-xdpyinfo
xfce4-session xfce4-panel xfdesktop xfwm4 gnome-terminal`。
滚动 Arch 的 FFmpeg 可能超出 `inproc-av` 支持的 6.x;请使用默认进程方案或独立的 FFmpeg 6 前缀,
不要降级系统 FFmpeg。
若编译包含 `viewer-ffmpeg`,滚动 Arch 的 FFmpeg 可能超出其支持的 6.x;请使用独立的 FFmpeg 6 前缀,不要降级系统 FFmpeg(服务端的 `--features inproc-av` 原生 NVENC 不依赖 FFmpeg)。
### 实验性 KMS 截屏
+131
View File
@@ -0,0 +1,131 @@
//! Standalone benchmark tool measuring:
//! 1. Host Memory Copy Bandwidth & CPU Cost (What FFmpeg/Sway CPU swizzling does)
//! 2. NVENC H.265 Hardware Throughput & Latency (1920x1200 @ ultra-low-latency)
//! 3. NVENC reading a registered (pinned) host buffer
use std::time::Instant;
fn main() {
println!("=========================================================");
println!(" UU Remote Linux: Encoder Pipeline Benchmark Suite ");
println!("=========================================================");
println!();
let width: u32 = 1920;
let height: u32 = 1200;
let frames: usize = 300; // 5 seconds of 60fps video
let frame_bytes: usize = (width * height * 4) as usize;
println!("Test Configuration:");
println!(" Resolution : {}x{}", width, height);
println!(" Frame Size : {:.2} MB (ARGB32)", frame_bytes as f64 / 1024.0 / 1024.0);
println!(" Iterations : {} frames (Equivalent to 5.0s @ 60 FPS)", frames);
println!();
// Simulating XShm framebuffer
let mut host_pixels = vec![0x55u8; frame_bytes];
// Benchmark 1: CPU Memcpy (External FFmpeg stdin / Sway wlr-screencopy CPU copy baseline)
println!("[1] Baseline: CPU Frame Memcpy Overhead (IPC / Process Pipe Cost)");
let mut sink = vec![0u8; frame_bytes];
let start_memcpy = Instant::now();
for _ in 0..frames {
sink.copy_from_slice(&host_pixels);
std::hint::black_box(&sink);
}
let duration_memcpy = start_memcpy.elapsed();
let memcpy_per_frame = duration_memcpy.as_secs_f64() * 1000.0 / frames as f64;
let memcpy_gbps = (frame_bytes * frames) as f64 / duration_memcpy.as_secs_f64() / 1e9;
let cpu_load_60fps = (memcpy_per_frame / (1000.0 / 60.0)) * 100.0;
println!(" -> Total CPU copy time : {:.2?}", duration_memcpy);
println!(" -> Per-frame copy time : {:.3} ms", memcpy_per_frame);
println!(" -> Memory Bus Bandwidth : {:.2} GB/s", memcpy_gbps);
println!(" -> CPU Core Saturation : {:.1}% of 1 core consumed solely by memcpy at 60 FPS", cpu_load_60fps);
println!();
// Benchmark 2: in-process NVENC reading a registered host buffer (device copy when registration fails)
#[cfg(feature = "inproc-av")]
{
extern "C" {
fn nvenc_create(config: *const NvencConfigFFI) -> *mut std::ffi::c_void;
fn nvenc_register_host_buffer(
engine: *mut std::ffi::c_void,
host_ptr: *mut std::ffi::c_void,
size: usize,
) -> i32;
fn nvenc_encode_frame(
engine: *mut std::ffi::c_void,
input_argb: *const u8,
pitch: u32,
force_idr: i32,
out_data: *mut *const u8,
out_len: *mut u32,
) -> i32;
fn nvenc_destroy(engine: *mut std::ffi::c_void);
}
// Must match NvencConfig in c_engine/nvenc_engine.h.
#[repr(C)]
struct NvencConfigFFI {
width: u32,
height: u32,
fps: u32,
gop: u32,
cq: u32,
target_bps: i64,
max_bps: i64,
buffer_bits: i64,
is_hevc: i32,
}
let cfg = NvencConfigFFI {
width,
height,
fps: 60,
gop: 120,
cq: 18,
target_bps: 16_000_000,
max_bps: 30_000_000,
buffer_bits: 500_000,
is_hevc: 1, // H.265 / HEVC
};
println!("[2] In-process NVENC H.265 encoding");
unsafe {
let engine = nvenc_create(&cfg);
if engine.is_null() {
println!(" [SKIP] NVENC driver not available on this system.");
} else {
let registered = nvenc_register_host_buffer(engine, host_pixels.as_mut_ptr() as *mut _, frame_bytes) == 0;
println!(" -> Input path : {}", if registered { "registered host buffer (DMA)" } else { "cuMemcpyHtoD copy" });
let input = if registered { std::ptr::null() } else { host_pixels.as_ptr() };
let mut out_data: *const u8 = std::ptr::null();
let mut out_len: u32 = 0;
// Warmup
let _ = nvenc_encode_frame(engine, input, width * 4, 1, &mut out_data, &mut out_len);
let start = Instant::now();
let (mut encoded_bytes, mut encoded_frames) = (0u64, 0usize);
for i in 0..frames {
if nvenc_encode_frame(engine, input, width * 4, i32::from(i == 0), &mut out_data, &mut out_len) == 0 {
encoded_bytes += u64::from(out_len);
encoded_frames += 1;
}
}
let elapsed = start.elapsed();
if encoded_frames == 0 {
println!(" [FAIL] no frame was encoded");
} else {
let per_frame = elapsed.as_secs_f64() * 1000.0 / encoded_frames as f64;
println!(" -> Encoded frames : {encoded_frames}/{frames} in {elapsed:.2?}");
println!(" -> Per-frame latency : {per_frame:.3} ms ({:.1} FPS max)", 1000.0 / per_frame);
println!(" -> Output bitrate : {:.2} Mbps average", encoded_bytes as f64 * 8.0 / elapsed.as_secs_f64() / 1e6);
if registered {
println!(" -> Host-to-device copy : none ({:.0} MB/s at 60 fps avoided)", frame_bytes as f64 * 60.0 / 1024.0 / 1024.0);
}
}
nvenc_destroy(engine);
}
}
}
}
+12 -4
View File
@@ -1,6 +1,10 @@
fn main() {
println!("cargo:rerun-if-changed=c_engine/portal_clipboard.inc");
println!("cargo:rerun-if-changed=c_engine/x11_engine.c");
println!("cargo:rerun-if-changed=c_engine/nvenc_engine.c");
println!("cargo:rerun-if-changed=c_engine/nvenc_engine.h");
println!("cargo:rerun-if-changed=vendor/ffnvcodec/ffnvcodec/dynlink_cuda.h");
println!("cargo:rerun-if-changed=vendor/ffnvcodec/ffnvcodec/nvEncodeAPI.h");
println!("cargo:rerun-if-changed=c_engine/clipboard.c");
println!("cargo:rerun-if-changed=c_engine/assist_window.c");
println!("cargo:rerun-if-changed=c_engine/viewer_window.c");
@@ -21,11 +25,15 @@ fn main() {
build.compile("uu_kms_engine");
println!("cargo:rustc-link-lib=drm");
}
cc::Build::new()
.file("c_engine/x11_engine.c")
let mut x11_build = cc::Build::new();
x11_build.file("c_engine/x11_engine.c")
.file("c_engine/clipboard.c")
.file("c_engine/pam_check.c")
.compile("x11_engine");
.file("c_engine/pam_check.c");
if std::env::var_os("CARGO_FEATURE_INPROC_AV").is_some() {
x11_build.file("c_engine/nvenc_engine.c");
x11_build.include("vendor/ffnvcodec");
}
x11_build.compile("x11_engine");
println!("cargo:rustc-link-lib=X11");
println!("cargo:rustc-link-lib=Xext");
+565
View File
@@ -0,0 +1,565 @@
#include "nvenc_engine.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include <ffnvcodec/dynlink_cuda.h>
#include <ffnvcodec/nvEncodeAPI.h>
// Struct revisions come from the vendored nvEncodeAPI.h (*_VER macros), so the session must be
// opened with that header's NVENCAPI_VERSION; older drivers are rejected in nvenc_create.
#define CU_MEMHOSTREGISTER_DEVICEMAP 0x02
typedef CUresult CUDAAPI (*tcuMemHostRegister)(void *p, size_t bytesize, unsigned int Flags);
typedef CUresult CUDAAPI (*tcuMemHostGetDevicePointer)(CUdeviceptr *pdptr, void *p, unsigned int Flags);
typedef CUresult CUDAAPI (*tcuMemHostUnregister)(void *p);
typedef NVENCSTATUS NVENCAPI (*tNvEncodeAPICreateInstance)(NV_ENCODE_API_FUNCTION_LIST *functionList);
typedef NVENCSTATUS NVENCAPI (*tNvEncodeAPIGetMaxSupportedVersion)(uint32_t *version);
struct NvencEngine {
void *cuda_lib;
void *nvenc_lib;
tcuInit *cuInit;
tcuDeviceGet *cuDeviceGet;
tcuDevicePrimaryCtxRetain *cuDevicePrimaryCtxRetain;
tcuDevicePrimaryCtxRelease *cuDevicePrimaryCtxRelease;
tcuCtxCreate_v2 *cuCtxCreate;
tcuCtxDestroy_v2 *cuCtxDestroy;
tcuCtxPushCurrent_v2 *cuCtxPushCurrent;
tcuCtxPopCurrent_v2 *cuCtxPopCurrent;
tcuMemAlloc_v2 *cuMemAlloc;
tcuMemFree_v2 *cuMemFree;
tcuMemcpyHtoD_v2 *cuMemcpyHtoD;
tcuMemHostRegister cuMemHostRegister;
tcuMemHostGetDevicePointer cuMemHostGetDevicePointer;
tcuMemHostUnregister cuMemHostUnregister;
CUdevice dev;
CUcontext cu_ctx;
int is_standalone_ctx; // cuCtxCreate fallback (primary context unavailable, seen on GB10)
CUdeviceptr cu_dptr;
size_t frame_bytes;
// Pinned XShm buffer registered for direct DMA reads
void *registered_host_ptr;
int is_zero_copy;
NV_ENCODE_API_FUNCTION_LIST fn;
void *encoder;
NV_ENC_REGISTER_RESOURCE reg_res;
NV_ENC_OUTPUT_PTR bitstream_buf;
uint32_t width;
uint32_t height;
int is_hevc;
uint8_t *output_staging;
size_t output_staging_cap;
};
// Returns 1 when the context was pushed; pop only in that case.
static int ctx_push(NvencEngine *engine) {
return engine->cuCtxPushCurrent && engine->cu_ctx && engine->cuCtxPushCurrent(engine->cu_ctx) == CUDA_SUCCESS;
}
static void ctx_pop(NvencEngine *engine, int pushed) {
if (pushed && engine->cuCtxPopCurrent) {
CUcontext dummy = NULL;
engine->cuCtxPopCurrent(&dummy);
}
}
NvencEngine* nvenc_create(const NvencConfig* config) {
if (!config || config->width == 0 || config->height == 0) {
return NULL;
}
NvencEngine *engine = (NvencEngine*)calloc(1, sizeof(NvencEngine));
if (!engine) return NULL;
engine->width = config->width;
engine->height = config->height;
engine->is_hevc = config->is_hevc;
// 1. Load CUDA driver
engine->cuda_lib = dlopen("libcuda.so.1", RTLD_NOW);
if (!engine->cuda_lib) {
fprintf(stderr, "[nvenc] dlopen libcuda.so.1 failed\n");
free(engine);
return NULL;
}
engine->cuInit = (tcuInit*)dlsym(engine->cuda_lib, "cuInit");
engine->cuDeviceGet = (tcuDeviceGet*)dlsym(engine->cuda_lib, "cuDeviceGet");
engine->cuDevicePrimaryCtxRetain = (tcuDevicePrimaryCtxRetain*)dlsym(engine->cuda_lib, "cuDevicePrimaryCtxRetain");
engine->cuDevicePrimaryCtxRelease = (tcuDevicePrimaryCtxRelease*)dlsym(engine->cuda_lib, "cuDevicePrimaryCtxRelease_v2");
if (!engine->cuDevicePrimaryCtxRelease)
engine->cuDevicePrimaryCtxRelease = (tcuDevicePrimaryCtxRelease*)dlsym(engine->cuda_lib, "cuDevicePrimaryCtxRelease");
engine->cuCtxCreate = (tcuCtxCreate_v2*)dlsym(engine->cuda_lib, "cuCtxCreate_v2");
engine->cuCtxDestroy = (tcuCtxDestroy_v2*)dlsym(engine->cuda_lib, "cuCtxDestroy_v2");
engine->cuCtxPushCurrent = (tcuCtxPushCurrent_v2*)dlsym(engine->cuda_lib, "cuCtxPushCurrent_v2");
engine->cuCtxPopCurrent = (tcuCtxPopCurrent_v2*)dlsym(engine->cuda_lib, "cuCtxPopCurrent_v2");
engine->cuMemAlloc = (tcuMemAlloc_v2*)dlsym(engine->cuda_lib, "cuMemAlloc_v2");
engine->cuMemFree = (tcuMemFree_v2*)dlsym(engine->cuda_lib, "cuMemFree_v2");
engine->cuMemcpyHtoD = (tcuMemcpyHtoD_v2*)dlsym(engine->cuda_lib, "cuMemcpyHtoD_v2");
engine->cuMemHostRegister = (tcuMemHostRegister)dlsym(engine->cuda_lib, "cuMemHostRegister_v2");
engine->cuMemHostGetDevicePointer = (tcuMemHostGetDevicePointer)dlsym(engine->cuda_lib, "cuMemHostGetDevicePointer_v2");
engine->cuMemHostUnregister = (tcuMemHostUnregister)dlsym(engine->cuda_lib, "cuMemHostUnregister");
if (!engine->cuInit || !engine->cuDeviceGet || !engine->cuDevicePrimaryCtxRetain ||
!engine->cuDevicePrimaryCtxRelease || !engine->cuCtxPushCurrent || !engine->cuCtxPopCurrent ||
!engine->cuMemAlloc || !engine->cuMemFree || !engine->cuMemcpyHtoD) {
fprintf(stderr, "[nvenc] Missing CUDA driver symbols\n");
nvenc_destroy(engine);
return NULL;
}
CUresult cures = engine->cuInit(0);
if (cures != CUDA_SUCCESS) {
fprintf(stderr, "[nvenc] cuInit failed: %d\n", cures);
nvenc_destroy(engine);
return NULL;
}
cures = engine->cuDeviceGet(&engine->dev, 0);
if (cures != CUDA_SUCCESS) {
fprintf(stderr, "[nvenc] cuDeviceGet failed: %d\n", cures);
nvenc_destroy(engine);
return NULL;
}
cures = engine->cuDevicePrimaryCtxRetain(&engine->cu_ctx, engine->dev);
if ((cures != CUDA_SUCCESS || !engine->cu_ctx) && engine->cuCtxCreate && engine->cuCtxDestroy) {
engine->cu_ctx = NULL;
// cuCtxCreate makes the new context current; the push below makes it current again
cures = engine->cuCtxCreate(&engine->cu_ctx, 0, engine->dev);
if (cures == CUDA_SUCCESS && engine->cu_ctx) {
CUcontext dummy = NULL;
engine->cuCtxPopCurrent(&dummy);
engine->is_standalone_ctx = 1;
}
}
if (cures != CUDA_SUCCESS || !engine->cu_ctx) {
fprintf(stderr, "[nvenc] CUDA context creation failed: %d\n", cures);
engine->cu_ctx = NULL;
nvenc_destroy(engine);
return NULL;
}
int pushed = ctx_push(engine);
if (!pushed) {
fprintf(stderr, "[nvenc] cuCtxPushCurrent failed\n");
nvenc_destroy(engine);
return NULL;
}
// 2. Load NVENC
engine->nvenc_lib = dlopen("libnvidia-encode.so.1", RTLD_NOW);
if (!engine->nvenc_lib) {
fprintf(stderr, "[nvenc] dlopen libnvidia-encode.so.1 failed\n");
goto fail;
}
// Like FFmpeg: the driver must support the header's API version, otherwise fall back.
tNvEncodeAPIGetMaxSupportedVersion get_max_ver = (tNvEncodeAPIGetMaxSupportedVersion)dlsym(
engine->nvenc_lib, "NvEncodeAPIGetMaxSupportedVersion");
uint32_t max_supported_ver = 0;
if (!get_max_ver || get_max_ver(&max_supported_ver) != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] NvEncodeAPIGetMaxSupportedVersion failed\n");
goto fail;
}
if (max_supported_ver < ((NVENCAPI_MAJOR_VERSION << 4) | NVENCAPI_MINOR_VERSION)) {
fprintf(stderr, "[nvenc] Driver NVENC API %u.%u is older than required %d.%d; update the NVIDIA driver\n",
max_supported_ver >> 4, max_supported_ver & 0xf, NVENCAPI_MAJOR_VERSION, NVENCAPI_MINOR_VERSION);
goto fail;
}
tNvEncodeAPICreateInstance create_inst = (tNvEncodeAPICreateInstance)dlsym(engine->nvenc_lib, "NvEncodeAPICreateInstance");
if (!create_inst) {
fprintf(stderr, "[nvenc] dlsym NvEncodeAPICreateInstance failed\n");
goto fail;
}
memset(&engine->fn, 0, sizeof(engine->fn));
engine->fn.version = NV_ENCODE_API_FUNCTION_LIST_VER;
NVENCSTATUS nvres = create_inst(&engine->fn);
if (nvres != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] NvEncodeAPICreateInstance failed: %d\n", nvres);
goto fail;
}
// 3. Open Encode Session
NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS open_params;
memset(&open_params, 0, sizeof(open_params));
open_params.version = NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER;
open_params.deviceType = NV_ENC_DEVICE_TYPE_CUDA;
open_params.device = engine->cu_ctx;
open_params.apiVersion = NVENCAPI_VERSION;
nvres = engine->fn.nvEncOpenEncodeSessionEx(&open_params, &engine->encoder);
if (nvres != NV_ENC_SUCCESS || !engine->encoder) {
fprintf(stderr, "[nvenc] nvEncOpenEncodeSessionEx failed: %d\n", nvres);
engine->encoder = NULL;
goto fail;
}
// 4. Query Preset Config
GUID codec_guid = engine->is_hevc ? NV_ENC_CODEC_HEVC_GUID : NV_ENC_CODEC_H264_GUID;
NV_ENC_PRESET_CONFIG preset_cfg;
memset(&preset_cfg, 0, sizeof(preset_cfg));
preset_cfg.version = NV_ENC_PRESET_CONFIG_VER;
preset_cfg.presetCfg.version = NV_ENC_CONFIG_VER;
nvres = engine->fn.nvEncGetEncodePresetConfigEx(
engine->encoder,
codec_guid,
NV_ENC_PRESET_P4_GUID,
NV_ENC_TUNING_INFO_ULTRA_LOW_LATENCY,
&preset_cfg
);
if (nvres != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] nvEncGetEncodePresetConfigEx failed: %d\n", nvres);
goto fail;
}
NV_ENC_CONFIG enc_cfg;
memcpy(&enc_cfg, &preset_cfg.presetCfg, sizeof(NV_ENC_CONFIG));
enc_cfg.gopLength = config->gop > 0 ? config->gop : NVENC_INFINITE_GOPLENGTH;
enc_cfg.frameIntervalP = 1; // Strict low-latency: zero B-frames
enc_cfg.rcParams.rateControlMode = NV_ENC_PARAMS_RC_VBR;
enc_cfg.rcParams.averageBitRate = (uint32_t)(config->target_bps > 0 ? config->target_bps : 10000000);
enc_cfg.rcParams.maxBitRate = (uint32_t)(config->max_bps > 0 ? config->max_bps : 15000000);
enc_cfg.rcParams.vbvBufferSize = (uint32_t)(config->buffer_bits > 0 ? config->buffer_bits : (config->max_bps > 0 ? config->max_bps / 10 : 1500000));
enc_cfg.rcParams.targetQuality = (uint8_t)(config->cq > 0 ? config->cq : 28);
enc_cfg.rcParams.enableAQ = 1;
// Temporal AQ only where the hardware/codec reports it
NV_ENC_CAPS_PARAM caps_param;
memset(&caps_param, 0, sizeof(caps_param));
caps_param.version = NV_ENC_CAPS_PARAM_VER;
caps_param.capsToQuery = NV_ENC_CAPS_SUPPORT_TEMPORAL_AQ;
int caps_val = 0;
enc_cfg.rcParams.enableTemporalAQ =
engine->fn.nvEncGetEncodeCaps(engine->encoder, codec_guid, &caps_param, &caps_val) == NV_ENC_SUCCESS && caps_val > 0;
if (engine->is_hevc) {
enc_cfg.profileGUID = NV_ENC_HEVC_PROFILE_MAIN_GUID;
enc_cfg.encodeCodecConfig.hevcConfig.idrPeriod = enc_cfg.gopLength;
enc_cfg.encodeCodecConfig.hevcConfig.repeatSPSPPS = 1;
enc_cfg.encodeCodecConfig.hevcConfig.outputAUD = 1;
} else {
enc_cfg.profileGUID = NV_ENC_H264_PROFILE_BASELINE_GUID;
enc_cfg.encodeCodecConfig.h264Config.idrPeriod = enc_cfg.gopLength;
enc_cfg.encodeCodecConfig.h264Config.repeatSPSPPS = 1;
enc_cfg.encodeCodecConfig.h264Config.outputAUD = 1;
}
// 5. Initialize Encoder
NV_ENC_INITIALIZE_PARAMS init_params;
memset(&init_params, 0, sizeof(init_params));
init_params.version = NV_ENC_INITIALIZE_PARAMS_VER;
init_params.encodeGUID = codec_guid;
init_params.presetGUID = NV_ENC_PRESET_P4_GUID;
init_params.encodeWidth = engine->width;
init_params.encodeHeight = engine->height;
init_params.darWidth = engine->width;
init_params.darHeight = engine->height;
init_params.frameRateNum = config->fps > 0 ? config->fps : 60;
init_params.frameRateDen = 1;
init_params.enablePTD = 1;
init_params.tuningInfo = NV_ENC_TUNING_INFO_ULTRA_LOW_LATENCY;
init_params.encodeConfig = &enc_cfg;
nvres = engine->fn.nvEncInitializeEncoder(engine->encoder, &init_params);
if (nvres != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] nvEncInitializeEncoder failed: %d\n", nvres);
goto fail;
}
// 6. Device input buffer (ARGB32), used until a host buffer is registered
engine->frame_bytes = (size_t)engine->width * engine->height * 4;
cures = engine->cuMemAlloc(&engine->cu_dptr, engine->frame_bytes);
if (cures != CUDA_SUCCESS) {
fprintf(stderr, "[nvenc] cuMemAlloc failed: %d\n", cures);
engine->cu_dptr = 0;
goto fail;
}
memset(&engine->reg_res, 0, sizeof(engine->reg_res));
engine->reg_res.version = NV_ENC_REGISTER_RESOURCE_VER;
engine->reg_res.resourceType = NV_ENC_INPUT_RESOURCE_TYPE_CUDADEVICEPTR;
engine->reg_res.width = engine->width;
engine->reg_res.height = engine->height;
engine->reg_res.pitch = engine->width * 4;
engine->reg_res.resourceToRegister = (void*)engine->cu_dptr;
engine->reg_res.bufferFormat = NV_ENC_BUFFER_FORMAT_ARGB;
nvres = engine->fn.nvEncRegisterResource(engine->encoder, &engine->reg_res);
if (nvres != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] nvEncRegisterResource failed: %d\n", nvres);
engine->reg_res.registeredResource = NULL;
goto fail;
}
// 7. Create bitstream output buffer
NV_ENC_CREATE_BITSTREAM_BUFFER bs_buf;
memset(&bs_buf, 0, sizeof(bs_buf));
bs_buf.version = NV_ENC_CREATE_BITSTREAM_BUFFER_VER;
nvres = engine->fn.nvEncCreateBitstreamBuffer(engine->encoder, &bs_buf);
if (nvres != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] nvEncCreateBitstreamBuffer failed: %d\n", nvres);
goto fail;
}
engine->bitstream_buf = bs_buf.bitstreamBuffer;
engine->output_staging_cap = 2 * 1024 * 1024;
engine->output_staging = (uint8_t*)malloc(engine->output_staging_cap);
if (!engine->output_staging) {
fprintf(stderr, "[nvenc] output buffer allocation failed\n");
goto fail;
}
ctx_pop(engine, pushed);
return engine;
fail:
// Pop the creation push first; nvenc_destroy pushes and pops on its own.
ctx_pop(engine, pushed);
nvenc_destroy(engine);
return NULL;
}
int nvenc_register_host_buffer(NvencEngine* engine, void* host_ptr, size_t size) {
if (!engine || !host_ptr || size < engine->frame_bytes) {
return -1;
}
if (!engine->cuMemHostRegister || !engine->cuMemHostGetDevicePointer || !engine->cuMemHostUnregister) {
return -2;
}
int pushed = ctx_push(engine);
if (!pushed) return -2;
// If already registered, unregister first
nvenc_unregister_host_buffer(engine);
CUresult cures = engine->cuMemHostRegister(host_ptr, size, CU_MEMHOSTREGISTER_DEVICEMAP);
if (cures != CUDA_SUCCESS) {
fprintf(stderr, "[nvenc] cuMemHostRegister failed: %d\n", cures);
ctx_pop(engine, pushed);
return -3;
}
CUdeviceptr dev_ptr = 0;
cures = engine->cuMemHostGetDevicePointer(&dev_ptr, host_ptr, 0);
if (cures != CUDA_SUCCESS || dev_ptr == 0) {
fprintf(stderr, "[nvenc] cuMemHostGetDevicePointer failed: %d\n", cures);
engine->cuMemHostUnregister(host_ptr);
ctx_pop(engine, pushed);
return -4;
}
// Register the pinned buffer before releasing the device buffer, so a failure keeps the copy path working
NV_ENC_REGISTER_RESOURCE new_reg;
memset(&new_reg, 0, sizeof(new_reg));
new_reg.version = NV_ENC_REGISTER_RESOURCE_VER;
new_reg.resourceType = NV_ENC_INPUT_RESOURCE_TYPE_CUDADEVICEPTR;
new_reg.width = engine->width;
new_reg.height = engine->height;
new_reg.pitch = engine->width * 4;
new_reg.resourceToRegister = (void*)dev_ptr;
new_reg.bufferFormat = NV_ENC_BUFFER_FORMAT_ARGB;
NVENCSTATUS nvres = engine->fn.nvEncRegisterResource(engine->encoder, &new_reg);
if (nvres != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] nvEncRegisterResource (pinned host) failed: %d\n", nvres);
engine->cuMemHostUnregister(host_ptr);
ctx_pop(engine, pushed);
return -5;
}
if (engine->reg_res.registeredResource) {
engine->fn.nvEncUnregisterResource(engine->encoder, engine->reg_res.registeredResource);
}
if (engine->cu_dptr) {
engine->cuMemFree(engine->cu_dptr);
engine->cu_dptr = 0;
}
engine->reg_res = new_reg;
engine->registered_host_ptr = host_ptr;
engine->is_zero_copy = 1;
ctx_pop(engine, pushed);
return 0;
}
// Caller has the context current.
void nvenc_unregister_host_buffer(NvencEngine* engine) {
if (!engine || !engine->is_zero_copy) return;
if (engine->reg_res.registeredResource && engine->encoder) {
engine->fn.nvEncUnregisterResource(engine->encoder, engine->reg_res.registeredResource);
memset(&engine->reg_res, 0, sizeof(engine->reg_res));
}
if (engine->registered_host_ptr && engine->cuMemHostUnregister) {
engine->cuMemHostUnregister(engine->registered_host_ptr);
engine->registered_host_ptr = NULL;
}
engine->is_zero_copy = 0;
}
int nvenc_encode_frame(
NvencEngine* engine,
const uint8_t* input_argb,
uint32_t pitch,
int force_idr,
const uint8_t** out_data,
uint32_t* out_len
) {
if (!engine || !out_data || !out_len || !engine->reg_res.registeredResource) {
return -1;
}
*out_data = NULL;
*out_len = 0;
int pushed = ctx_push(engine);
if (!pushed) return -1;
// Without a registered host buffer, copy the frame into the device buffer
if (!engine->is_zero_copy) {
if (!input_argb) {
ctx_pop(engine, pushed);
return -1;
}
CUresult cures = CUDA_SUCCESS;
if (pitch == engine->width * 4) {
cures = engine->cuMemcpyHtoD(engine->cu_dptr, input_argb, engine->frame_bytes);
} else {
for (uint32_t y = 0; y < engine->height && cures == CUDA_SUCCESS; y++) {
cures = engine->cuMemcpyHtoD(
engine->cu_dptr + (size_t)y * engine->width * 4,
input_argb + (size_t)y * pitch,
engine->width * 4
);
}
}
if (cures != CUDA_SUCCESS) {
fprintf(stderr, "[nvenc] cuMemcpyHtoD failed: %d\n", cures);
ctx_pop(engine, pushed);
return -2;
}
}
NV_ENC_MAP_INPUT_RESOURCE map_res;
memset(&map_res, 0, sizeof(map_res));
map_res.version = NV_ENC_MAP_INPUT_RESOURCE_VER;
map_res.registeredResource = engine->reg_res.registeredResource;
NVENCSTATUS nvres = engine->fn.nvEncMapInputResource(engine->encoder, &map_res);
if (nvres != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] nvEncMapInputResource failed: %d\n", nvres);
ctx_pop(engine, pushed);
return -3;
}
NV_ENC_PIC_PARAMS pic_params;
memset(&pic_params, 0, sizeof(pic_params));
pic_params.version = NV_ENC_PIC_PARAMS_VER;
pic_params.inputWidth = engine->width;
pic_params.inputHeight = engine->height;
pic_params.inputPitch = engine->width * 4;
pic_params.inputBuffer = map_res.mappedResource;
pic_params.outputBitstream = engine->bitstream_buf;
pic_params.bufferFmt = NV_ENC_BUFFER_FORMAT_ARGB;
pic_params.pictureStruct = NV_ENC_PIC_STRUCT_FRAME;
if (force_idr) {
pic_params.encodePicFlags = NV_ENC_PIC_FLAG_FORCEIDR;
}
nvres = engine->fn.nvEncEncodePicture(engine->encoder, &pic_params);
if (nvres != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] nvEncEncodePicture failed: %d\n", nvres);
engine->fn.nvEncUnmapInputResource(engine->encoder, map_res.mappedResource);
ctx_pop(engine, pushed);
return -4;
}
NV_ENC_LOCK_BITSTREAM lock_bs;
memset(&lock_bs, 0, sizeof(lock_bs));
lock_bs.version = NV_ENC_LOCK_BITSTREAM_VER;
lock_bs.outputBitstream = engine->bitstream_buf;
lock_bs.doNotWait = 0;
nvres = engine->fn.nvEncLockBitstream(engine->encoder, &lock_bs);
if (nvres != NV_ENC_SUCCESS) {
fprintf(stderr, "[nvenc] nvEncLockBitstream failed: %d\n", nvres);
engine->fn.nvEncUnmapInputResource(engine->encoder, map_res.mappedResource);
ctx_pop(engine, pushed);
return -5;
}
int rc = 0;
uint32_t size = lock_bs.bitstreamSizeInBytes;
if (size > engine->output_staging_cap) {
size_t cap = (size_t)size + 64 * 1024;
uint8_t *grown = (uint8_t*)realloc(engine->output_staging, cap);
if (grown) {
engine->output_staging = grown;
engine->output_staging_cap = cap;
} else {
fprintf(stderr, "[nvenc] output buffer allocation failed: %u bytes\n", size);
rc = -6;
}
}
if (rc == 0 && size > 0 && lock_bs.bitstreamBufferPtr) {
memcpy(engine->output_staging, lock_bs.bitstreamBufferPtr, size);
*out_data = engine->output_staging;
*out_len = size;
}
engine->fn.nvEncUnlockBitstream(engine->encoder, engine->bitstream_buf);
// The SDK allows unmapping only after the bitstream lock returned for this picture
engine->fn.nvEncUnmapInputResource(engine->encoder, map_res.mappedResource);
ctx_pop(engine, pushed);
return rc;
}
void nvenc_destroy(NvencEngine* engine) {
if (!engine) return;
int pushed = ctx_push(engine);
// Without a current context the NVENC/CUDA objects cannot be released safely; leak them.
if (pushed) {
if (engine->bitstream_buf && engine->encoder) {
engine->fn.nvEncDestroyBitstreamBuffer(engine->encoder, engine->bitstream_buf);
engine->bitstream_buf = NULL;
}
nvenc_unregister_host_buffer(engine);
if (engine->reg_res.registeredResource && engine->encoder) {
engine->fn.nvEncUnregisterResource(engine->encoder, engine->reg_res.registeredResource);
memset(&engine->reg_res, 0, sizeof(engine->reg_res));
}
if (engine->cu_dptr) {
engine->cuMemFree(engine->cu_dptr);
engine->cu_dptr = 0;
}
if (engine->encoder) {
engine->fn.nvEncDestroyEncoder(engine->encoder);
engine->encoder = NULL;
}
ctx_pop(engine, pushed);
}
if (engine->cu_ctx) {
if (engine->is_standalone_ctx) engine->cuCtxDestroy(engine->cu_ctx);
else engine->cuDevicePrimaryCtxRelease(engine->dev);
engine->cu_ctx = NULL;
}
free(engine->output_staging);
// libcuda/libnvidia-encode stay loaded: their driver threads outlive the session, and
// dlclose crashed them on the next encoder start (SEGV_MAPERR in libnvcuvid).
free(engine);
}
+58
View File
@@ -0,0 +1,58 @@
#ifndef UU_NVENC_ENGINE_H
#define UU_NVENC_ENGINE_H
#include <stdint.h>
#include <stddef.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct NvencEngine NvencEngine;
typedef struct {
uint32_t width;
uint32_t height;
uint32_t fps;
uint32_t gop;
uint32_t cq;
int64_t target_bps;
int64_t max_bps;
int64_t buffer_bits;
int is_hevc; // 1 = H.265, 0 = H.264
} NvencConfig;
// Initialize and open an NVENC encoding session. Returns NULL on error.
NvencEngine* nvenc_create(const NvencConfig* config);
// Register host shared memory (XShm buffer) as CUDA pinned memory that NVENC reads by DMA.
// Returns 0 on success, negative value on error.
int nvenc_register_host_buffer(NvencEngine* engine, void* host_ptr, size_t size);
// Unregister a previously registered host buffer (the CUDA context must be current).
void nvenc_unregister_host_buffer(NvencEngine* engine);
// Encode one frame.
// If host buffer was registered via nvenc_register_host_buffer, input_argb can be NULL
// and the encoder reads the registered host memory directly (no host-to-device copy).
// force_idr: 1 to force keyframe/IDR, 0 otherwise
// out_data: pointer to receive the bitstream buffer (valid until next encode call or destroy)
// out_len: pointer to receive the bitstream size in bytes
// Returns 0 on success, negative value on error.
int nvenc_encode_frame(
NvencEngine* engine,
const uint8_t* input_argb,
uint32_t pitch,
int force_idr,
const uint8_t** out_data,
uint32_t* out_len
);
// Destroy the NVENC engine and free all CUDA/NVENC resources.
void nvenc_destroy(NvencEngine* engine);
#ifdef __cplusplus
}
#endif
#endif // UU_NVENC_ENGINE_H
+1 -10
View File
@@ -1,7 +1,6 @@
//! Desktop sound for the viewer's 声音 menu: the default output's monitor (PulseAudio or
//! PipeWire-pulse) is encoded to Opus and sent on the negotiated audio m-line.
//! Encoding runs in this process with the `inproc-av` feature (unless `UU_FFMPEG_ROOT` is set),
//! otherwise an FFmpeg child process does it.
//! An FFmpeg child process does the encoding.
//! PipeWire monitors ignore the sink's mute by default, so 被控端静音运行 keeps the stream audible.
//! `UU_AUDIO=0` disables it; `UU_AUDIO_SOURCE` picks another Pulse source.
@@ -57,14 +56,6 @@ fn read_packets(stdout: impl Read, tx: &tokio::sync::mpsc::Sender<bytes::Bytes>)
fn start(source: &str, tx: tokio::sync::mpsc::Sender<bytes::Bytes>)
-> Result<(Stopper, tokio::task::JoinHandle<Result<(), String>>), String> {
let stop = Arc::new(AtomicBool::new(false));
#[cfg(feature = "inproc-av")]
if crate::inproc_encoder::selected() {
let job = tokio::task::spawn_blocking({
let (stop, source) = (stop.clone(), source.to_owned());
move || crate::inproc_encoder::run_audio(&source, tx, stop)
});
return Ok((Stopper { stop, child: None }, job));
}
let mut child = spawn_encoder(source)?;
let stdout = child.stdout.take().ok_or("Encoder stdout missing")?;
let job = tokio::task::spawn_blocking(move || read_packets(stdout, &tx));
+10
View File
@@ -70,6 +70,16 @@ impl DamageCapture {
pub fn size(&self) -> (u32, u32) { (self.width as u32, self.height as u32) }
pub fn shm_raw_ptr(&self) -> *mut u8 { self.shm_ptr }
pub fn shm_size(&self) -> usize { self.len }
pub fn refresh(&mut self) -> Result<(), String> {
self.conn.shm_get_image(self.root, 0, 0, self.width, self.height, !0, Z_PIXMAP, self.seg, 0)
.map_err(text)?.reply().map_err(text)?;
Ok(())
}
/// Waits up to `timeout` for the screen to change. Returns `true` on change, `false` on timeout.
pub fn wait(&mut self, timeout: Duration) -> Result<bool, String> {
let deadline = Instant::now() + timeout;
+397 -399
View File
@@ -1,21 +1,64 @@
//! In-process X11 capture and H.264/HEVC encoding through libavdevice/libavcodec.
//! It replaces the `ffmpeg` child process with the same codec options, so the viewer
//! still receives the same Annex-B access units (AUD, SPS/PPS, IDR, P).
use std::ffi::CString;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Once};
use std::time::{Duration, Instant};
use ffmpeg_next as ff;
use ff::format::{format::Format, sample, Pixel, Sample};
use ff::ChannelLayout;
use ff::util::picture;
use ff::{codec, encoder, frame, software::scaling, Dictionary, Packet, Rational};
use tokio::sync::mpsc;
use tokio::sync::{Semaphore, OwnedSemaphorePermit};
//! In-process X11 capture (XDamage + XShm) and NVENC encoding through the NVIDIA driver.
//! When capture and encoder sizes match, the XShm buffer is registered as CUDA pinned memory and
//! NVENC reads it by DMA, so there is no host-to-device copy (the X server still writes XShm).
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::{mpsc, OwnedSemaphorePermit, Semaphore};
use crate::damage_capture::DamageCapture;
#[repr(C)]
struct NvencEngineOpaque {
_private: [u8; 0],
}
#[repr(C)]
struct NvencConfigFFI {
width: u32,
height: u32,
fps: u32,
gop: u32,
cq: u32,
target_bps: i64,
max_bps: i64,
buffer_bits: i64,
is_hevc: libc::c_int,
}
extern "C" {
fn nvenc_create(config: *const NvencConfigFFI) -> *mut NvencEngineOpaque;
fn nvenc_register_host_buffer(
engine: *mut NvencEngineOpaque,
host_ptr: *mut libc::c_void,
size: usize,
) -> libc::c_int;
fn nvenc_encode_frame(
engine: *mut NvencEngineOpaque,
input_argb: *const u8,
pitch: u32,
force_idr: libc::c_int,
out_data: *mut *const u8,
out_len: *mut u32,
) -> libc::c_int;
fn nvenc_destroy(engine: *mut NvencEngineOpaque);
}
struct SafeNvencEngine(*mut NvencEngineOpaque);
unsafe impl Send for SafeNvencEngine {}
impl Drop for SafeNvencEngine {
fn drop(&mut self) {
if !self.0.is_null() {
unsafe { nvenc_destroy(self.0) };
self.0 = std::ptr::null_mut();
}
}
}
pub struct Config {
pub display: String,
pub input: (u32, u32),
@@ -26,43 +69,52 @@ pub struct Config {
pub target_bps: i64,
pub max_bps: i64,
pub buffer_bits: i64,
pub encoder: crate::video::Encoder,
pub codec: crate::capture_setting::VideoCodec,
/// Capture on XDamage changes (XShm) instead of x11grab; `UU_CAPTURE=grab` turns it off.
pub damage: bool,
}
/// `UU_VIDEO_PIPELINE=process` keeps the original ffmpeg child-process path.
pub fn disabled() -> bool {
std::env::var("UU_VIDEO_PIPELINE").is_ok_and(|v| v == "process")
}
/// Damage-driven capture is on unless `UU_CAPTURE=grab` asks for x11grab.
pub fn damage_enabled() -> bool {
std::env::var("UU_CAPTURE").map_or(true, |v| v != "grab")
}
/// In-process capture is used unless the process path is requested, or a bundled FFmpeg prefix
/// (`UU_FFMPEG_ROOT` / config `ffmpeg_root`) is set: that build was chosen on purpose, so keep using it.
pub fn selected() -> bool {
!disabled() && crate::config::ffmpeg_root().is_none()
pub fn selected(encoder: crate::video::Encoder) -> bool {
encoder == crate::video::Encoder::Nvenc && damage_enabled() && !disabled() && crate::config::ffmpeg_root().is_none()
}
fn text(error: ff::Error) -> String { error.to_string() }
fn ensure_init() -> Result<(), String> {
static INIT: Once = Once::new();
let mut result = Ok(());
INIT.call_once(|| {
if let Err(error) = ff::init() { result = Err(text(error)); }
ff::device::register_all();
});
result
/// One encoder thread per stream. Aborting the reader only asks the blocking thread to stop at its
/// next wake-up, so a restart waits here until the previous NVENC session and XShm are released.
/// The key is the stream's IDR flag, which survives encoder restarts.
pub async fn stream_slot(stream: &Arc<AtomicBool>) -> tokio::sync::OwnedMutexGuard<()> {
type Slots = Vec<(std::sync::Weak<AtomicBool>, Arc<tokio::sync::Mutex<()>>)>;
static SLOTS: std::sync::Mutex<Slots> = std::sync::Mutex::new(Vec::new());
let slot = {
let mut slots = SLOTS.lock().unwrap_or_else(|e| e.into_inner());
slots.retain(|(key, _)| key.strong_count() > 0);
match slots.iter().find(|(key, _)| std::ptr::eq(key.as_ptr(), Arc::as_ptr(stream))) {
Some((_, slot)) => slot.clone(),
None => {
let slot = Arc::new(tokio::sync::Mutex::new(()));
slots.push((Arc::downgrade(stream), slot.clone()));
slot
}
}
};
slot.lock_owned().await
}
pub struct EncodedUnit { pub data: Vec<u8>, pub permit: OwnedSemaphorePermit, pub completed_at: Instant }
pub struct EncodedUnit {
pub data: Vec<u8>,
pub permit: OwnedSemaphorePermit,
pub completed_at: Instant,
}
fn acquire(budget: &Arc<Semaphore>, units: &mpsc::Sender<EncodedUnit>) -> Option<OwnedSemaphorePermit> {
fn acquire(
budget: &Arc<Semaphore>,
units: &mpsc::Sender<EncodedUnit>,
) -> Option<OwnedSemaphorePermit> {
tokio::runtime::Handle::current().block_on(async {
tokio::select! {
permit = budget.clone().acquire_owned() => permit.ok(),
@@ -71,294 +123,6 @@ fn acquire(budget: &Arc<Semaphore>, units: &mpsc::Sender<EncodedUnit>) -> Option
})
}
#[cfg(test)]
mod budget_tests {
use super::*;
#[tokio::test]
async fn closing_encoder_output_wakes_a_blocked_capture() {
let budget=Arc::new(Semaphore::new(0));
let (tx,rx)=mpsc::channel(1);
let worker=tokio::task::spawn_blocking(move || acquire(&budget,&tx));
tokio::task::yield_now().await;
drop(rx);
assert!(tokio::time::timeout(Duration::from_secs(1),worker).await.unwrap().unwrap().is_none());
}
}
fn drain(encoder: &mut encoder::Video, packet: &mut Packet, units: &mpsc::Sender<EncodedUnit>, permits: &mut VecDeque<OwnedSemaphorePermit>) -> Result<bool, String> {
loop {
match encoder.receive_packet(packet) {
Ok(()) => {
let data = packet.data().unwrap_or_default().to_vec();
let permit = permits.pop_front().ok_or("Encoder returned an unbudgeted access unit")?;
if units.blocking_send(EncodedUnit {data, permit, completed_at: Instant::now()}).is_err() { return Ok(false); }
}
Err(ff::Error::Other { errno }) if errno == libc::EAGAIN => return Ok(true),
Err(error) => return Err(text(error)),
}
}
}
/// Opens the selected video encoder for `config`. Returns the encoder, its input pixel format and output size.
fn open_encoder(config: &Config) -> Result<(encoder::Video, Pixel, u32, u32), String> {
// H.264/HEVC 4:2:0 and NVENC need even dimensions; an odd native capture is padded (see run_damage).
let (width, height) = config.output.unwrap_or(config.input);
let (width, height) = (width + width % 2, height + height % 2);
let codec = encoder::find_by_name(config.encoder.name_for(config.codec))
.ok_or_else(|| format!("{} is not available in this FFmpeg build", config.encoder.name_for(config.codec)))?;
// Allocate for this codec so FFmpeg fills its defaults; libx264 rejects a context allocated without one.
let mut context = unsafe { codec::context::Context::wrap(ff::sys::avcodec_alloc_context3(codec.as_ptr()), None) }.encoder().video().map_err(text)?;
context.set_width(width);
context.set_height(height);
// NVENC takes packed BGR0 directly, so the GPU path skips the CPU colour conversion to YUV420P.
let pixel = match config.encoder { crate::video::Encoder::Nvenc => Pixel::BGRZ, crate::video::Encoder::Vaapi => Pixel::NV12, _ => Pixel::YUV420P };
context.set_format(if config.encoder == crate::video::Encoder::Vaapi { Pixel::VAAPI } else { pixel });
if config.encoder == crate::video::Encoder::Vaapi { attach_vaapi(&mut context, width, height)?; }
context.set_time_base(Rational::new(1, config.fps as i32));
context.set_frame_rate(Some(Rational::new(config.fps as i32, 1)));
context.set_gop(config.gop);
context.set_max_b_frames(0);
context.set_bit_rate(config.target_bps as usize);
context.set_max_bit_rate(config.max_bps as usize);
// The CLI's -maxrate/-bufsize map to these raw fields, which the encoder wrappers read.
unsafe {
let raw = context.as_mut_ptr();
(*raw).rc_max_rate = config.max_bps;
(*raw).rc_buffer_size = config.buffer_bits as i32;
}
let cq = config.cq.to_string();
let gop = config.gop.to_string();
let mut options = Dictionary::new();
let hevc = config.codec == crate::capture_setting::VideoCodec::H265;
if config.encoder == crate::video::Encoder::Vaapi {
for (key,value) in [("profile", if hevc { "main" } else { "constrained_baseline" }), ("rc_mode", "VBR"), ("async_depth", "1"), ("aud", "1")] { options.set(key,value); }
options.set("qmin", &cq);
} else if config.encoder == crate::video::Encoder::Nvenc {
for (key, value) in [("preset", "p4"), ("tune", "ull"), ("profile", if hevc { "main" } else { "baseline" }), ("rc", "vbr"),
("rc-lookahead", "0"), ("zerolatency", "1"), ("delay", "0"), ("aud", "1"), ("forced-idr", "1")] {
options.set(key, value);
}
options.set("cq", &cq);
} else if hevc {
for (key, value) in [("preset", "superfast"), ("tune", "zerolatency"), ("profile", "main"), ("threads", "4"), ("forced-idr", "1")] {
options.set(key, value);
}
options.set("crf", &cq);
options.set("x265-params", &crate::video::x265_params(&gop));
} else {
let x264 = format!("aud=1:repeat-headers=1:scenecut=0:keyint={gop}:min-keyint={gop}");
for (key, value) in [("preset", "superfast"), ("tune", "zerolatency"), ("profile", "baseline"), ("threads", "4")] {
options.set(key, value);
}
options.set("crf", &cq);
options.set("x264-params", &x264);
}
let encoder = context.open_as_with(codec, options).map_err(text)?;
Ok((encoder, pixel, width, height))
}
// Each codec owns a reference to its hardware frame pool. Every error path releases
// the temporary device/pool refs; AVCodecContext releases its own ref on drop.
fn attach_vaapi(context: &mut encoder::video::Video, width: u32, height: u32) -> Result<(), String> {
use ff::sys::*;
let device_name = CString::new(crate::video::vaapi_device()).map_err(|_| "Invalid VAAPI device")?;
unsafe {
let mut device = std::ptr::null_mut();
let result = av_hwdevice_ctx_create(&mut device, AVHWDeviceType::AV_HWDEVICE_TYPE_VAAPI, device_name.as_ptr(), std::ptr::null_mut(), 0);
if result < 0 { return Err(format!("VAAPI device initialization failed: {}", text(ff::Error::from(result)))); }
let mut pool = av_hwframe_ctx_alloc(device);
av_buffer_unref(&mut device);
if pool.is_null() { return Err("VAAPI frame pool allocation failed".into()); }
let frames = (*pool).data as *mut AVHWFramesContext;
(*frames).format = AVPixelFormat::AV_PIX_FMT_VAAPI;
(*frames).sw_format = AVPixelFormat::AV_PIX_FMT_NV12;
(*frames).width = width as i32;
(*frames).height = height as i32;
(*frames).initial_pool_size = 8;
let result = av_hwframe_ctx_init(pool);
if result < 0 { av_buffer_unref(&mut pool); return Err(format!("VAAPI frame pool initialization failed: {}", text(ff::Error::from(result)))); }
(*context.as_mut_ptr()).hw_frames_ctx = pool;
}
Ok(())
}
fn send_video_frame(encoder: &mut encoder::Video, frame: &frame::Video, kind: crate::video::Encoder) -> Result<(), String> {
if kind != crate::video::Encoder::Vaapi { return encoder.send_frame(frame).map_err(text); }
let mut uploaded = frame::Video::empty();
unsafe {
let result = ff::sys::av_hwframe_get_buffer((*encoder.as_ptr()).hw_frames_ctx, uploaded.as_mut_ptr(), 0);
if result < 0 { return Err(text(ff::Error::from(result))); }
let result = ff::sys::av_hwframe_transfer_data(uploaded.as_mut_ptr(), frame.as_ptr(), 0);
if result < 0 { return Err(text(ff::Error::from(result))); }
let result = ff::sys::av_frame_copy_props(uploaded.as_mut_ptr(), frame.as_ptr());
if result < 0 { return Err(text(ff::Error::from(result))); }
}
encoder.send_frame(&uploaded).map_err(text)
}
/// Runs until `stop` is set or the receiver is dropped. A set `idr` flag forces the next frame to be an IDR.
pub fn run(config: Config, units: mpsc::Sender<EncodedUnit>, stop: Arc<AtomicBool>, idr: Arc<AtomicBool>, budget: Arc<Semaphore>) -> Result<(), String> {
ensure_init()?;
if config.damage {
match DamageCapture::open(&config.display) {
Ok(capture) => return run_damage(config, capture, units, stop, idr, budget),
Err(error) => log::warn!("Damage-driven capture unavailable, using x11grab: {error}"),
}
}
let name = CString::new("x11grab").map_err(|e| e.to_string())?;
let raw = unsafe { ff::sys::av_find_input_format(name.as_ptr()) };
if raw.is_null() { return Err("x11grab input is not available in this FFmpeg build".into()); }
let input_format = Format::Input(unsafe { ff::format::format::Input::wrap(raw as *mut _) });
let mut capture_options = Dictionary::new();
capture_options.set("framerate", &config.fps.to_string());
capture_options.set("video_size", &format!("{}x{}", config.input.0, config.input.1));
// The viewer draws its own pointer; do not bake a second one into the video.
capture_options.set("draw_mouse", "0");
let mut input = ff::format::open_with(&config.display, &input_format, capture_options).map_err(text)?.input();
let stream = input.streams().best(ff::media::Type::Video).ok_or("X11 capture has no video stream")?;
let index = stream.index();
let mut decoder = codec::context::Context::from_parameters(stream.parameters()).map_err(text)?.decoder().video().map_err(text)?;
let (mut encoder, pixel, width, height) = open_encoder(&config)?;
let mut scaler: Option<(scaling::Context, (Pixel, u32, u32))> = None;
let mut decoded = frame::Video::empty();
let mut converted = frame::Video::empty();
let mut packet = Packet::empty();
let started = Instant::now();
let frame_interval = Duration::from_nanos(1_000_000_000 / u64::from(config.fps));
let mut last_at: Option<Duration> = None;
let mut last_slot: i64 = -1;
let mut permits = VecDeque::with_capacity(4);
let mut packets = input.packets();
loop {
if stop.load(Ordering::Relaxed) { return Ok(()); }
let Some(permit) = acquire(&budget, &units) else { return Ok(()); };
let Some((stream, input_packet)) = packets.next() else { return Ok(()); };
if stream.index() != index { continue; }
decoder.send_packet(&input_packet).map_err(text)?;
match decoder.receive_frame(&mut decoded) {
Ok(()) => {}
Err(ff::Error::Other { errno }) if errno == libc::EAGAIN => continue,
Err(error) => return Err(text(error)),
}
// x11grab catches up after a stall by grabbing back-to-back. A frame arriving within half
// a frame interval of the previous one is part of that burst and is dropped, as the old
// `-fps_mode vfr` did. Normal jitter stays well above that threshold.
let now = started.elapsed();
if last_at.is_some_and(|previous| now - previous < frame_interval / 2) { continue; }
last_at = Some(now);
let slot = ((now.as_micros() * u128::from(config.fps) + 500_000) / 1_000_000) as i64;
let slot = slot.max(last_slot + 1);
last_slot = slot;
// Pass the capture buffer straight through when it already matches the encoder input.
let frame = if decoded.format() == pixel && decoded.width() == width && decoded.height() == height {
&mut decoded
} else {
let key = (decoded.format(), decoded.width(), decoded.height());
if scaler.as_ref().map(|(_, k)| *k) != Some(key) {
let context = scaling::Context::get(key.0, key.1, key.2, pixel, width, height, scaling::Flags::BILINEAR).map_err(text)?;
scaler = Some((context, key));
}
if let Some((context, _)) = scaler.as_mut() { context.run(&decoded, &mut converted).map_err(text)?; }
&mut converted
};
frame.set_pts(Some(slot));
frame.set_kind(if idr.swap(false, Ordering::AcqRel) { picture::Type::I } else { picture::Type::None });
send_video_frame(&mut encoder, frame, config.encoder)?;
permits.push_back(permit);
if !drain(&mut encoder, &mut packet, &units, &mut permits)? { return Ok(()); }
}
}
/// Damage-driven loop. Damage only marks the picture dirty; a dirty picture is captured and encoded on
/// the next fixed frame tick, so the stream keeps a steady cadence while the screen moves. The viewer's
/// jitter buffer grows with irregular arrivals, and irregular sends were what raised its latency.
fn run_damage(config: Config, mut capture: DamageCapture, units: mpsc::Sender<EncodedUnit>, stop: Arc<AtomicBool>, idr: Arc<AtomicBool>, budget: Arc<Semaphore>) -> Result<(), String> {
let (mut encoder, pixel, width, height) = open_encoder(&config)?;
let (cap_width, cap_height) = capture.size();
// Native size uses the even encoder size. The extra row/column is never overwritten, so it stays black.
let native = config.output.is_none();
let (frame_width, frame_height) = if native { (width, height) } else { (cap_width, cap_height) };
let mut captured = frame::Video::new(Pixel::BGRZ, frame_width, frame_height);
if native { captured.data_mut(0).fill(0); }
let mut converted = frame::Video::empty();
let mut scaler = None;
let mut packet = Packet::empty();
let started = Instant::now();
let frame_interval = Duration::from_nanos(1_000_000_000 / u64::from(config.fps));
// Keepalive gaps are random between 15 and 25 fps: a steady 1 fps reads as a stall to the viewer.
let mut seed = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(1, |d| d.subsec_nanos()) | 1;
let mut keepalive = random_keepalive(&mut seed);
let mut dirty = true;
let mut next_tick = Instant::now();
let mut last_send = Instant::now();
let mut last_slot: i64 = -1;
let mut permits = VecDeque::with_capacity(4);
let locked = crate::session_lock::watch_locked();
let mut last_locked = *locked.borrow();
loop {
if stop.load(Ordering::Relaxed) { return Ok(()); }
let deadline = if dirty { next_tick } else { last_send + keepalive };
let wait = deadline.saturating_duration_since(Instant::now());
// A change marks the picture dirty and falls through, so a client that damages the root constantly
// cannot keep an overdue tick waiting.
if capture.wait(wait.max(Duration::from_millis(1)))? { dirty = true; }
let now = Instant::now();
if dirty && now < next_tick { continue; }
if !dirty && now < last_send + keepalive { continue; }
let Some(permit) = acquire(&budget, &units) else { return Ok(()); };
if stop.load(Ordering::Relaxed) { return Ok(()); }
dirty |= capture.wait(Duration::ZERO)?;
let state = *locked.borrow();
if state != last_locked { dirty = true; last_locked = state; }
if dirty {
let source = capture.capture()?;
let bytes_per_line = cap_width as usize * 4;
let stride = captured.stride(0);
let row = bytes_per_line.min(stride);
for (y, src) in source.chunks_exact(bytes_per_line).enumerate() {
captured.data_mut(0)[y * stride..y * stride + row].copy_from_slice(&src[..row]);
}
}
let slot = ((started.elapsed().as_micros() * u128::from(config.fps) + 500_000) / 1_000_000) as i64;
let slot = slot.max(last_slot + 1);
last_slot = slot;
let frame = if pixel == Pixel::BGRZ && width == captured.width() && height == captured.height() {
&mut captured
} else {
if scaler.is_none() {
scaler = Some(scaling::Context::get(Pixel::BGRZ, captured.width(), captured.height(), pixel, width, height, scaling::Flags::BILINEAR).map_err(text)?);
}
if dirty {
if let Some(context) = scaler.as_mut() { context.run(&captured, &mut converted).map_err(text)?; }
}
&mut converted
};
frame.set_pts(Some(slot));
frame.set_kind(if idr.swap(false, Ordering::AcqRel) { picture::Type::I } else { picture::Type::None });
send_video_frame(&mut encoder, frame, config.encoder)?;
permits.push_back(permit);
if !drain(&mut encoder, &mut packet, &units, &mut permits)? { return Ok(()); }
let now = Instant::now();
if dirty {
// Keep the tick phase; after a long idle, restart the phase from now instead of bursting to catch up.
let next = next_tick + frame_interval;
next_tick = if next > now { next } else { now + frame_interval };
dirty = false;
}
last_send = now;
keepalive = random_keepalive(&mut seed);
}
}
/// Picks the next keepalive gap in 40–67 ms (15–25 fps) with a small xorshift generator.
fn random_keepalive(seed: &mut u32) -> Duration {
*seed ^= *seed << 13;
*seed ^= *seed >> 17;
@@ -366,89 +130,323 @@ fn random_keepalive(seed: &mut u32) -> Duration {
Duration::from_micros(40_000 + u64::from(*seed % 26_700))
}
/// Encodes a PulseAudio/PipeWire source to Opus in this process. Each Opus packet is sent as is.
pub fn run_audio(source: &str, units: mpsc::Sender<bytes::Bytes>, stop: Arc<AtomicBool>) -> Result<(), String> {
ensure_init()?;
let name = CString::new("pulse").map_err(|e| e.to_string())?;
let raw = unsafe { ff::sys::av_find_input_format(name.as_ptr()) };
if raw.is_null() { return Err("pulse input is not available in this FFmpeg build".into()); }
let input_format = Format::Input(unsafe { ff::format::format::Input::wrap(raw as *mut _) });
let mut capture_options = Dictionary::new();
capture_options.set("fragment_size", "1920");
let mut input = ff::format::open_with(&source, &input_format, capture_options).map_err(text)?.input();
pub fn run(
config: Config,
units: mpsc::Sender<EncodedUnit>,
stop: Arc<AtomicBool>,
idr: Arc<AtomicBool>,
budget: Arc<Semaphore>,
) -> Result<(), String> {
let (width, height) = config.output.unwrap_or(config.input);
let (width, height) = (width + width % 2, height + height % 2);
let is_hevc = if config.codec == crate::capture_setting::VideoCodec::H265 {
1
} else {
0
};
let stream = input.streams().best(ff::media::Type::Audio).ok_or("Pulse source has no audio stream")?;
let index = stream.index();
let mut decoder = codec::context::Context::from_parameters(stream.parameters()).map_err(text)?.decoder().audio().map_err(text)?;
log::info!(
"Initializing Native In-Process NVENC: {}x{}, {} fps, GOP {}, CQ {}, HEVC={}",
width,
height,
config.fps,
config.gop,
config.cq,
is_hevc == 1
);
let codec = encoder::find_by_name("libopus").ok_or("Opus encoder is not available in this FFmpeg build")?;
let mut context = unsafe { codec::context::Context::wrap(ff::sys::avcodec_alloc_context3(codec.as_ptr()), None) }.encoder().audio().map_err(text)?;
context.set_rate(48_000);
context.set_format(Sample::I16(sample::Type::Packed));
// FFmpeg 6 reads the AVChannelLayout field; the deprecated channel_layout alone makes libopus fail with EINVAL.
unsafe { ff::sys::av_channel_layout_default(&mut (*context.as_mut_ptr()).ch_layout, 2) };
context.set_time_base(Rational::new(1, 48_000));
context.set_bit_rate(128_000);
let mut options = Dictionary::new();
options.set("application", "lowdelay");
options.set("frame_duration", "20");
let mut encoder = context.open_as_with(codec, options).map_err(text)?;
// libopus takes fixed-size frames: 960 samples per channel for 20 ms.
let frame_samples = match encoder.frame_size() as usize { 0 => 960, n => n };
let ffi_cfg = NvencConfigFFI {
width,
height,
fps: config.fps,
gop: config.gop,
cq: config.cq,
target_bps: config.target_bps,
max_bps: config.max_bps,
buffer_bits: config.buffer_bits,
is_hevc,
};
let mut resampler = None;
let mut decoded = frame::Audio::empty();
let mut converted = frame::Audio::empty();
let mut fifo: Vec<i16> = Vec::new();
let mut pts: i64 = 0;
let mut packet = Packet::empty();
for (stream, input_packet) in input.packets() {
if stop.load(Ordering::Relaxed) { return Ok(()); }
if stream.index() != index { continue; }
decoder.send_packet(&input_packet).map_err(text)?;
loop {
match decoder.receive_frame(&mut decoded) {
Ok(()) => {}
Err(ff::Error::Other { errno }) if errno == libc::EAGAIN => break,
Err(error) => return Err(text(error)),
let mut capture = DamageCapture::open(&config.display)
.map_err(|e| format!("DamageCapture::open failed on {}: {e}", config.display))?;
let raw_engine = unsafe { nvenc_create(&ffi_cfg) };
if raw_engine.is_null() {
return Err("Failed to create native NVENC engine".into());
}
let engine = SafeNvencEngine(raw_engine);
let (cap_width, cap_height) = capture.size();
// Check if capture buffer can be Zero-Copy registered directly
let mut zero_copy = false;
if cap_width == width && cap_height == height {
let shm_ptr = capture.shm_raw_ptr() as *mut libc::c_void;
let shm_len = capture.shm_size();
let res = unsafe { nvenc_register_host_buffer(engine.0, shm_ptr, shm_len) };
if res == 0 {
zero_copy = true;
log::info!("Native NVENC reads the registered XShm buffer directly (no host-to-device copy)");
} else {
log::warn!("nvenc_register_host_buffer returned {res}, copying frames with cuMemcpyHtoD");
}
} else {
log::info!(
"Capture size ({}x{}) != encoder size ({}x{}): {} on the CPU, then cuMemcpyHtoD",
cap_width,
cap_height,
width,
height,
if config.output.is_some() { "bilinear scaling" } else { "padding" }
);
}
let bytes_per_line = cap_width as usize * 4;
let frame_interval = Duration::from_nanos(1_000_000_000 / u64::from(config.fps));
let mut seed = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map_or(1, |d| d.subsec_nanos())
| 1;
let mut keepalive = random_keepalive(&mut seed);
let mut dirty = true;
let mut next_tick = Instant::now();
let mut last_send = Instant::now();
let mut permits = VecDeque::with_capacity(4);
let mut frame_buf = Vec::new();
let locked = crate::session_lock::watch_locked();
let mut last_locked = *locked.borrow();
loop {
if stop.load(Ordering::Relaxed) {
return Ok(());
}
let deadline = if dirty {
next_tick
} else {
last_send + keepalive
};
let wait = deadline.saturating_duration_since(Instant::now());
if capture.wait(wait.max(Duration::from_millis(1)))? {
dirty = true;
}
let now = Instant::now();
if dirty && now < next_tick {
continue;
}
if !dirty && now < last_send + keepalive {
continue;
}
let Some(permit) = acquire(&budget, &units) else {
return Ok(());
};
if stop.load(Ordering::Relaxed) {
return Ok(());
}
dirty |= capture.wait(Duration::ZERO)?;
let state = *locked.borrow();
if state != last_locked {
dirty = true;
last_locked = state;
}
let force_idr = if idr.swap(false, Ordering::AcqRel) {
1
} else {
0
};
let mut out_data: *const u8 = std::ptr::null();
let mut out_len: u32 = 0;
let res = if zero_copy {
// In Zero-Copy mode: refresh XShm, then NVENC DMA-reads directly from shm_ptr
if dirty {
capture.refresh()?;
}
// The decoder may leave the deprecated channel_layout unset; derive it from the channel count.
let source_layout = ChannelLayout::default(i32::from(decoded.channels()));
if resampler.is_none() {
resampler = Some(ff::software::resampling::Context::get(
decoded.format(), source_layout, decoded.rate(),
Sample::I16(sample::Type::Packed), ChannelLayout::STEREO, 48_000,
).map_err(text)?);
unsafe {
nvenc_encode_frame(
engine.0,
std::ptr::null(), // NULL means use zero-copy registered buffer
bytes_per_line as u32,
force_idr,
&mut out_data,
&mut out_len,
)
}
// Keep both layout fields of the decoded frame in step with the resampler's input layout.
decoded.set_channel_layout(source_layout);
unsafe { ff::sys::av_channel_layout_default(&mut (*decoded.as_mut_ptr()).ch_layout, i32::from(decoded.channels())) };
if let Some(resampler) = resampler.as_mut() { resampler.run(&decoded, &mut converted).map_err(text)?; }
let samples = converted.samples() * 2;
fifo.extend(converted.data(0)[..samples * 2].chunks_exact(2).map(|b| i16::from_ne_bytes([b[0], b[1]])));
} else {
// Copy path: 其他分辨率 is scaled; an odd native size is padded with black to the even encoder size
let source = capture.capture()?;
let (target_ptr, pitch) = if cap_width == width && cap_height == height {
(source.as_ptr(), bytes_per_line as u32)
} else {
frame_buf.resize(width as usize * height as usize * 4, 0);
if config.output.is_some() {
scale_bilinear_argb32(source, cap_width, cap_height, &mut frame_buf, width, height);
} else {
pad_argb32(source, cap_width, cap_height, &mut frame_buf, width, height);
}
(frame_buf.as_ptr(), width * 4)
};
unsafe {
nvenc_encode_frame(
engine.0,
target_ptr,
pitch,
force_idr,
&mut out_data,
&mut out_len,
)
}
};
while fifo.len() >= frame_samples * 2 {
let mut out = frame::Audio::new(Sample::I16(sample::Type::Packed), frame_samples, ChannelLayout::STEREO);
unsafe { ff::sys::av_channel_layout_default(&mut (*out.as_mut_ptr()).ch_layout, 2) };
out.set_rate(48_000);
out.set_pts(Some(pts));
for (dst, src) in out.data_mut(0)[..frame_samples * 4].chunks_exact_mut(2).zip(fifo.drain(..frame_samples * 2)) {
dst.copy_from_slice(&src.to_ne_bytes());
}
pts += frame_samples as i64;
encoder.send_frame(&out).map_err(text)?;
loop {
match encoder.receive_packet(&mut packet) {
Ok(()) => {
let data = bytes::Bytes::copy_from_slice(packet.data().unwrap_or_default());
if units.blocking_send(data).is_err() { return Ok(()); }
}
Err(ff::Error::Other { errno }) if errno == libc::EAGAIN => break,
Err(error) => return Err(text(error)),
}
}
if res != 0 {
return Err(format!("nvenc_encode_frame failed with error {res}"));
}
if out_len > 0 && !out_data.is_null() {
let data = unsafe { std::slice::from_raw_parts(out_data, out_len as usize) }.to_vec();
permits.push_back(permit);
let send_permit = permits.pop_front().unwrap();
if units
.blocking_send(EncodedUnit {
data,
permit: send_permit,
completed_at: Instant::now(),
})
.is_err()
{
return Ok(());
}
}
let now = Instant::now();
if dirty {
let next = next_tick + frame_interval;
next_tick = if next > now {
next
} else {
now + frame_interval
};
dirty = false;
}
last_send = now;
keepalive = random_keepalive(&mut seed);
}
}
/// Bilinear ARGB32 scaling (the former swscale path used SWS_BILINEAR), with 8-bit fixed-point
/// weights and pixel centers aligned.
fn scale_bilinear_argb32(src: &[u8], src_w: u32, src_h: u32, dst: &mut [u8], dst_w: u32, dst_h: u32) {
if src_w == 0 || src_h == 0 || dst_w == 0 || dst_h == 0 {
return;
}
// Source position in 1/256 pixel units: (d + 0.5) * src / dst - 0.5, clamped to the image.
let taps = |dst_len: u32, src_len: u32| -> Vec<(usize, usize, u32)> {
(0..dst_len)
.map(|d| {
let pos = ((2 * u64::from(d) + 1) * u64::from(src_len) * 128 / u64::from(dst_len)).saturating_sub(128);
let first = ((pos >> 8) as usize).min(src_len as usize - 1);
let second = (first + 1).min(src_len as usize - 1);
(first, second, (pos & 255) as u32)
})
.collect()
};
let (columns, rows) = (taps(dst_w, src_w), taps(dst_h, src_h));
let src_stride = src_w as usize * 4;
for (dst_row, &(y0, y1, fy)) in dst.chunks_exact_mut(dst_w as usize * 4).zip(&rows) {
let (top, bottom) = (&src[y0 * src_stride..], &src[y1 * src_stride..]);
for (pixel, &(x0, x1, fx)) in dst_row.chunks_exact_mut(4).zip(&columns) {
for c in 0..4 {
let lerp = |row: &[u8]| u32::from(row[x0 * 4 + c]) * (256 - fx) + u32::from(row[x1 * 4 + c]) * fx;
pixel[c] = ((lerp(top) * (256 - fy) + lerp(bottom) * fy + 32768) >> 16) as u8;
}
}
}
}
/// Copies the capture into the top-left corner of the larger encoder frame; the rest stays black.
fn pad_argb32(src: &[u8], src_w: u32, src_h: u32, dst: &mut [u8], dst_w: u32, dst_h: u32) {
let (src_stride, dst_stride) = (src_w as usize * 4, dst_w as usize * 4);
let copy = src_stride.min(dst_stride);
for (y, dst_row) in dst.chunks_exact_mut(dst_stride).enumerate() {
if y < src_h.min(dst_h) as usize {
dst_row[..copy].copy_from_slice(&src[y * src_stride..y * src_stride + copy]);
dst_row[copy..].fill(0);
} else {
dst_row.fill(0);
}
}
}
#[cfg(test)]
mod budget_tests {
use super::*;
#[tokio::test]
async fn closing_encoder_output_wakes_a_blocked_capture() {
let budget = Arc::new(Semaphore::new(0));
let (tx, rx) = mpsc::channel(1);
let worker = tokio::task::spawn_blocking(move || acquire(&budget, &tx));
tokio::task::yield_now().await;
drop(rx);
assert!(tokio::time::timeout(Duration::from_secs(1), worker).await.unwrap().unwrap().is_none());
}
#[tokio::test]
async fn stream_slot_waits_for_the_previous_encoder_of_the_same_stream() {
let (stream, other) = (Arc::new(AtomicBool::new(false)), Arc::new(AtomicBool::new(false)));
let held = stream_slot(&stream).await;
// Another stream is independent.
drop(tokio::time::timeout(Duration::from_secs(1), stream_slot(&other)).await.unwrap());
assert!(tokio::time::timeout(Duration::from_millis(50), stream_slot(&stream)).await.is_err());
drop(held);
drop(tokio::time::timeout(Duration::from_secs(1), stream_slot(&stream)).await.unwrap());
}
}
#[cfg(test)]
mod frame_tests {
use super::*;
fn image(w: u32, h: u32, pixel: impl Fn(u32, u32) -> [u8; 4]) -> Vec<u8> {
let pixel = &pixel;
(0..h).flat_map(|y| (0..w).flat_map(move |x| pixel(x, y))).collect()
}
#[test]
fn bilinear_scaling_keeps_flat_areas_and_averages_edges() {
let flat = image(8, 6, |_, _| [10, 20, 30, 255]);
let mut out = vec![0; 4 * 3 * 4];
scale_bilinear_argb32(&flat, 8, 6, &mut out, 4, 3);
assert!(out.chunks(4).all(|p| p == [10, 20, 30, 255]));
// A 2:1 downscale of alternating black/white columns lands between the two, not on one of them.
let stripes = image(4, 2, |x, _| if x % 2 == 0 { [0; 4] } else { [255; 4] });
let mut out = vec![0; 2 * 1 * 4];
scale_bilinear_argb32(&stripes, 4, 2, &mut out, 2, 1);
assert!(out.iter().all(|&v| v == 128), "{out:?}");
// Upscaling interpolates and never reads past the last row/column.
let ramp = image(2, 1, |x, _| [(x * 200) as u8; 4]);
let mut out = vec![0; 4 * 2 * 4];
scale_bilinear_argb32(&ramp, 2, 1, &mut out, 4, 2);
let row: Vec<u8> = out[..16].chunks(4).map(|p| p[0]).collect();
assert_eq!(row, vec![0, 50, 150, 200]);
}
#[test]
fn odd_capture_is_padded_with_black_not_stretched() {
let src = image(3, 3, |x, y| [(x + 3 * y + 1) as u8; 4]);
let mut out = vec![9; 4 * 4 * 4];
pad_argb32(&src, 3, 3, &mut out, 4, 4);
for y in 0..4usize {
for x in 0..4usize {
let expected = if x < 3 && y < 3 { (x + 3 * y + 1) as u8 } else { 0 };
assert_eq!(out[(y * 4 + x) * 4], expected, "({x},{y})");
}
}
}
Ok(())
}
+27 -9
View File
@@ -202,7 +202,7 @@ fn nvidia_present() -> bool {
/// in-process encoder are 4:2:0 only).
pub fn full_chroma_available() -> bool {
#[cfg(feature = "inproc-av")]
if crate::inproc_encoder::selected() { return false; }
if crate::inproc_encoder::selected(Encoder::requested()) { return false; }
Encoder::requested() != Encoder::Vaapi
}
@@ -506,17 +506,33 @@ async fn read_encoder(
) -> Result<(), String> {
// The sequence is shared so a CPU fallback or settings restart continues it (skip statistics stay correct).
let encoder = Encoder::requested();
if encoder == Encoder::Cpu { return read_encoder_with(size,display,portal,latest,Encoder::Cpu,sequence,settings,idr,budget).await; }
match read_encoder_with(size,display.clone(),portal.clone(),latest.clone(),encoder,sequence.clone(),settings,idr.clone(),budget.clone()).await {
if encoder == Encoder::Cpu { return read_encoder_with(size,display,portal,latest,Encoder::Cpu,true,sequence,settings,idr,budget).await; }
#[cfg_attr(not(feature = "inproc-av"), allow(unused_mut))]
let mut result = read_encoder_with(size,display.clone(),portal.clone(),latest.clone(),encoder,true,sequence.clone(),settings,idr.clone(),budget.clone()).await;
// A failed native NVENC session (old driver, unsupported mode) still leaves FFmpeg's NVENC process pipeline.
#[cfg(feature = "inproc-av")]
if let Err(error) = &result {
if !latest.is_closed() && native_inproc(&display, portal.is_some(), encoder) {
log::warn!("In-process {} stream unavailable; switching to the FFmpeg process pipeline: {}",encoder.name_for(settings.codec),error);
result = read_encoder_with(size,display.clone(),portal.clone(),latest.clone(),encoder,false,sequence.clone(),settings,idr.clone(),budget.clone()).await;
}
}
match result {
Ok(())=>Ok(()),
Err(error) if !latest.is_closed()=>{
log::warn!("{} stream unavailable; switching scoped encoder to CPU: {}",encoder.name_for(settings.codec),error);
read_encoder_with(size,display,portal,latest,Encoder::Cpu,sequence,settings,idr,budget).await
read_encoder_with(size,display,portal,latest,Encoder::Cpu,true,sequence,settings,idr,budget).await
},
Err(error)=>Err(error),
}
}
/// The native in-process NVENC path serves X11 only (not Wayland/KMS capture or a portal session).
#[cfg(feature = "inproc-av")]
fn native_inproc(display: &str, portal: bool, encoder: Encoder) -> bool {
!display.starts_with("wayland:") && !display.starts_with("kms:") && !portal && crate::inproc_encoder::selected(encoder)
}
/// Capture and encode inside this process; each packet is one access unit.
#[cfg(feature = "inproc-av")]
async fn read_inprocess(
@@ -528,9 +544,11 @@ async fn read_inprocess(
let stop = Arc::new(AtomicBool::new(false));
// If this future is dropped (reader task aborted), the blocking encoder stops at its next packet.
let _stop_on_drop = StopOnDrop(stop.clone());
// A restart aborts the previous reader without waiting for its blocking encoder; wait for it here.
let slot = crate::inproc_encoder::stream_slot(&idr).await;
let job = tokio::task::spawn_blocking({
let stop = stop.clone();
move || crate::inproc_encoder::run(config, units_tx, stop, idr, budget)
move || { let _slot = slot; crate::inproc_encoder::run(config, units_tx, stop, idr, budget) }
});
let mut gate = DependencyGate::default();
while let Some(unit) = units.recv().await {
@@ -579,7 +597,8 @@ fn rate_args(settings: &VideoSettings, encoder: Encoder) -> (String, String) {
}
async fn read_encoder_with(
size: (u32,u32),display:String,portal:Option<Arc<crate::portal::PortalSession>>,latest:mpsc::Sender<Arc<CompleteFrame>>,encoder:Encoder,sequence:Arc<std::sync::atomic::AtomicU64>,
size: (u32,u32),display:String,portal:Option<Arc<crate::portal::PortalSession>>,latest:mpsc::Sender<Arc<CompleteFrame>>,encoder:Encoder,
#[cfg_attr(not(feature = "inproc-av"), allow(unused_variables))] allow_inproc: bool,sequence:Arc<std::sync::atomic::AtomicU64>,
settings: VideoSettings, #[cfg_attr(not(feature = "inproc-av"), allow(unused_variables))] idr: Arc<AtomicBool>, budget: Arc<Semaphore>,
)->Result<(),String>{
let selected_output=if display.starts_with("wayland:") {crate::wayland::source(settings.source_id)} else {None};
@@ -593,14 +612,13 @@ async fn read_encoder_with(
Some(Arc::new(capture))
} else { None };
#[cfg(feature = "inproc-av")]
if !wayland && portal.is_none() && crate::inproc_encoder::selected() {
if allow_inproc && native_inproc(&display, portal.is_some(), encoder) {
let max_kbps = settings.max_kbps().unwrap_or(30_000) as i64;
let config = crate::inproc_encoder::Config {
display: display.clone(), input: size, output: settings.output, fps: u32::from(settings.fps),
gop: gop_frames(settings.fps, true) as u32, cq: settings.cq() as u32,
target_bps: settings.max_kbps().map_or(16_000, |k| i64::from(k) * 3 / 4) * 1000,
max_bps: max_kbps * 1000, buffer_bits: i64::from(buffer_for(&settings, encoder)) * 1000, encoder, codec: settings.codec,
damage: crate::inproc_encoder::damage_enabled(),
max_bps: max_kbps * 1000, buffer_bits: i64::from(buffer_for(&settings, encoder)) * 1000, codec: settings.codec,
};
log::info!("Desktop encoder: display={} source={}x{} output={:?} fps={} cq={} maxrate={}k encoder={} pipeline=in-process",
display,size.0,size.1,settings.output,settings.fps,config.cq,max_kbps,encoder.name_for(settings.codec));
+1 -1
View File
@@ -1,4 +1,4 @@
//! Uses the same optional FFmpeg dependency as the host inproc-av feature.
//! Uses the optional FFmpeg 6 libraries of the `viewer-ffmpeg` feature.
use super::desktop::{Decoder,Frame};
use ffmpeg_next as av;
use super::media::Codec;
+4
View File
@@ -0,0 +1,4 @@
`ffnvcodec/dynlink_cuda.h` and `ffnvcodec/nvEncodeAPI.h` are unmodified copies from
[nv-codec-headers](https://github.com/FFmpeg/nv-codec-headers) tag `n12.1.14.0`
(NVENC API 12.1, the version FFmpeg 6.1 builds against; NVIDIA Linux driver 530.41 or newer).
MIT licensed, see the header comments. Only the in-process NVENC engine (`--features inproc-av`) uses them.
+518
View File
@@ -0,0 +1,518 @@
/*
* This copyright notice applies to this header file only:
*
* Copyright (c) 2016
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the software, and to permit persons to whom the
* software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*/
#if !defined(FFNV_DYNLINK_CUDA_H) && !defined(CUDA_VERSION)
#define FFNV_DYNLINK_CUDA_H
#include <stddef.h>
#include <stdint.h>
#define CUDA_VERSION 7050
#if defined(_WIN32) || defined(__CYGWIN__)
#define CUDAAPI __stdcall
#else
#define CUDAAPI
#endif
#define CU_CTX_SCHED_BLOCKING_SYNC 4
typedef int CUdevice;
#if defined(__x86_64) || defined(AMD64) || defined(_M_AMD64) || defined(__LP64__) || defined(__aarch64__)
typedef unsigned long long CUdeviceptr;
#else
typedef unsigned int CUdeviceptr;
#endif
typedef unsigned long long CUtexObject;
typedef struct CUarray_st *CUarray;
typedef struct CUctx_st *CUcontext;
typedef struct CUstream_st *CUstream;
typedef struct CUevent_st *CUevent;
typedef struct CUfunc_st *CUfunction;
typedef struct CUmod_st *CUmodule;
typedef struct CUmipmappedArray_st *CUmipmappedArray;
typedef struct CUgraphicsResource_st *CUgraphicsResource;
typedef struct CUextMemory_st *CUexternalMemory;
typedef struct CUextSemaphore_st *CUexternalSemaphore;
typedef struct CUeglStreamConnection_st *CUeglStreamConnection;
typedef struct CUlinkState_st *CUlinkState;
typedef enum cudaError_enum {
CUDA_SUCCESS = 0,
CUDA_ERROR_NOT_READY = 600,
CUDA_ERROR_LAUNCH_TIMEOUT = 702,
CUDA_ERROR_UNKNOWN = 999
} CUresult;
/**
* Device properties (subset)
*/
typedef enum CUdevice_attribute_enum {
CU_DEVICE_ATTRIBUTE_CLOCK_RATE = 13,
CU_DEVICE_ATTRIBUTE_TEXTURE_ALIGNMENT = 14,
CU_DEVICE_ATTRIBUTE_MULTIPROCESSOR_COUNT = 16,
CU_DEVICE_ATTRIBUTE_INTEGRATED = 18,
CU_DEVICE_ATTRIBUTE_CAN_MAP_HOST_MEMORY = 19,
CU_DEVICE_ATTRIBUTE_COMPUTE_MODE = 20,
CU_DEVICE_ATTRIBUTE_CONCURRENT_KERNELS = 31,
CU_DEVICE_ATTRIBUTE_PCI_BUS_ID = 33,
CU_DEVICE_ATTRIBUTE_PCI_DEVICE_ID = 34,
CU_DEVICE_ATTRIBUTE_TCC_DRIVER = 35,
CU_DEVICE_ATTRIBUTE_MEMORY_CLOCK_RATE = 36,
CU_DEVICE_ATTRIBUTE_GLOBAL_MEMORY_BUS_WIDTH = 37,
CU_DEVICE_ATTRIBUTE_ASYNC_ENGINE_COUNT = 40,
CU_DEVICE_ATTRIBUTE_UNIFIED_ADDRESSING = 41,
CU_DEVICE_ATTRIBUTE_PCI_DOMAIN_ID = 50,
CU_DEVICE_ATTRIBUTE_TEXTURE_PITCH_ALIGNMENT = 51,
CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MAJOR = 75,
CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MINOR = 76,
CU_DEVICE_ATTRIBUTE_MANAGED_MEMORY = 83,
CU_DEVICE_ATTRIBUTE_MULTI_GPU_BOARD = 84,
CU_DEVICE_ATTRIBUTE_MULTI_GPU_BOARD_GROUP_ID = 85,
} CUdevice_attribute;
typedef enum CUarray_format_enum {
CU_AD_FORMAT_UNSIGNED_INT8 = 0x01,
CU_AD_FORMAT_UNSIGNED_INT16 = 0x02,
CU_AD_FORMAT_UNSIGNED_INT32 = 0x03,
CU_AD_FORMAT_SIGNED_INT8 = 0x08,
CU_AD_FORMAT_SIGNED_INT16 = 0x09,
CU_AD_FORMAT_SIGNED_INT32 = 0x0a,
CU_AD_FORMAT_HALF = 0x10,
CU_AD_FORMAT_FLOAT = 0x20
} CUarray_format;
typedef enum CUmemorytype_enum {
CU_MEMORYTYPE_HOST = 1,
CU_MEMORYTYPE_DEVICE = 2,
CU_MEMORYTYPE_ARRAY = 3
} CUmemorytype;
typedef enum CUlimit_enum {
CU_LIMIT_STACK_SIZE = 0,
CU_LIMIT_PRINTF_FIFO_SIZE = 1,
CU_LIMIT_MALLOC_HEAP_SIZE = 2,
CU_LIMIT_DEV_RUNTIME_SYNC_DEPTH = 3,
CU_LIMIT_DEV_RUNTIME_PENDING_LAUNCH_COUNT = 4
} CUlimit;
typedef enum CUresourcetype_enum {
CU_RESOURCE_TYPE_ARRAY = 0x00,
CU_RESOURCE_TYPE_MIPMAPPED_ARRAY = 0x01,
CU_RESOURCE_TYPE_LINEAR = 0x02,
CU_RESOURCE_TYPE_PITCH2D = 0x03
} CUresourcetype;
typedef enum CUaddress_mode_enum {
CU_TR_ADDRESS_MODE_WRAP = 0,
CU_TR_ADDRESS_MODE_CLAMP = 1,
CU_TR_ADDRESS_MODE_MIRROR = 2,
CU_TR_ADDRESS_MODE_BORDER = 3
} CUaddress_mode;
typedef enum CUfilter_mode_enum {
CU_TR_FILTER_MODE_POINT = 0,
CU_TR_FILTER_MODE_LINEAR = 1
} CUfilter_mode;
typedef enum CUgraphicsRegisterFlags_enum {
CU_GRAPHICS_REGISTER_FLAGS_NONE = 0,
CU_GRAPHICS_REGISTER_FLAGS_READ_ONLY = 1,
CU_GRAPHICS_REGISTER_FLAGS_WRITE_DISCARD = 2,
CU_GRAPHICS_REGISTER_FLAGS_SURFACE_LDST = 4,
CU_GRAPHICS_REGISTER_FLAGS_TEXTURE_GATHER = 8
} CUgraphicsRegisterFlags;
typedef enum CUexternalMemoryHandleType_enum {
CU_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD = 1,
CU_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32 = 2,
CU_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_KMT = 3,
CU_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_HEAP = 4,
CU_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_RESOURCE = 5,
} CUexternalMemoryHandleType;
typedef enum CUexternalSemaphoreHandleType_enum {
CU_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_FD = 1,
CU_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_WIN32 = 2,
CU_EXTERNAL_SEMAPHORE_HANDLE_TYPE_OPAQUE_WIN32_KMT = 3,
CU_EXTERNAL_SEMAPHORE_HANDLE_TYPE_D3D12_FENCE = 4,
CU_EXTERNAL_SEMAPHORE_HANDLE_TYPE_TIMELINE_SEMAPHORE_FD = 9,
CU_EXTERNAL_SEMAPHORE_HANDLE_TYPE_TIMELINE_SEMAPHORE_WIN32 = 10,
} CUexternalSemaphoreHandleType;
typedef enum CUjit_option_enum
{
CU_JIT_MAX_REGISTERS = 0,
CU_JIT_THREADS_PER_BLOCK = 1,
CU_JIT_WALL_TIME = 2,
CU_JIT_INFO_LOG_BUFFER = 3,
CU_JIT_INFO_LOG_BUFFER_SIZE_BYTES = 4,
CU_JIT_ERROR_LOG_BUFFER = 5,
CU_JIT_ERROR_LOG_BUFFER_SIZE_BYTES = 6,
CU_JIT_OPTIMIZATION_LEVEL = 7,
CU_JIT_TARGET_FROM_CUCONTEXT = 8,
CU_JIT_TARGET = 9,
CU_JIT_FALLBACK_STRATEGY = 10,
CU_JIT_GENERATE_DEBUG_INFO = 11,
CU_JIT_LOG_VERBOSE = 12,
CU_JIT_GENERATE_LINE_INFO = 13,
CU_JIT_CACHE_MODE = 14,
CU_JIT_NEW_SM3X_OPT = 15,
CU_JIT_FAST_COMPILE = 16,
CU_JIT_GLOBAL_SYMBOL_NAMES = 17,
CU_JIT_GLOBAL_SYMBOL_ADDRESSES = 18,
CU_JIT_GLOBAL_SYMBOL_COUNT = 19,
CU_JIT_NUM_OPTIONS
} CUjit_option;
typedef enum CUjitInputType_enum
{
CU_JIT_INPUT_CUBIN = 0,
CU_JIT_INPUT_PTX = 1,
CU_JIT_INPUT_FATBINARY = 2,
CU_JIT_INPUT_OBJECT = 3,
CU_JIT_INPUT_LIBRARY = 4,
CU_JIT_NUM_INPUT_TYPES
} CUjitInputType;
typedef enum CUeglFrameType
{
CU_EGL_FRAME_TYPE_ARRAY = 0,
CU_EGL_FRAME_TYPE_PITCH = 1,
} CUeglFrameType;
typedef enum CUeglColorFormat
{
CU_EGL_COLOR_FORMAT_YUV420_PLANAR = 0x00,
CU_EGL_COLOR_FORMAT_YUV420_SEMIPLANAR = 0x01,
CU_EGL_COLOR_FORMAT_YVU420_SEMIPLANAR = 0x15,
CU_EGL_COLOR_FORMAT_Y10V10U10_420_SEMIPLANAR = 0x17,
CU_EGL_COLOR_FORMAT_Y12V12U12_420_SEMIPLANAR = 0x19,
} CUeglColorFormat;
typedef enum CUd3d11DeviceList_enum
{
CU_D3D11_DEVICE_LIST_ALL = 1,
CU_D3D11_DEVICE_LIST_CURRENT_FRAME = 2,
CU_D3D11_DEVICE_LIST_NEXT_FRAME = 3,
} CUd3d11DeviceList;
#ifndef CU_UUID_HAS_BEEN_DEFINED
#define CU_UUID_HAS_BEEN_DEFINED
typedef struct CUuuid_st {
char bytes[16];
} CUuuid;
#endif
typedef struct CUDA_MEMCPY2D_st {
size_t srcXInBytes;
size_t srcY;
CUmemorytype srcMemoryType;
const void *srcHost;
CUdeviceptr srcDevice;
CUarray srcArray;
size_t srcPitch;
size_t dstXInBytes;
size_t dstY;
CUmemorytype dstMemoryType;
void *dstHost;
CUdeviceptr dstDevice;
CUarray dstArray;
size_t dstPitch;
size_t WidthInBytes;
size_t Height;
} CUDA_MEMCPY2D;
typedef struct CUDA_RESOURCE_DESC_st {
CUresourcetype resType;
union {
struct {
CUarray hArray;
} array;
struct {
CUmipmappedArray hMipmappedArray;
} mipmap;
struct {
CUdeviceptr devPtr;
CUarray_format format;
unsigned int numChannels;
size_t sizeInBytes;
} linear;
struct {
CUdeviceptr devPtr;
CUarray_format format;
unsigned int numChannels;
size_t width;
size_t height;
size_t pitchInBytes;
} pitch2D;
struct {
int reserved[32];
} reserved;
} res;
unsigned int flags;
} CUDA_RESOURCE_DESC;
typedef struct CUDA_TEXTURE_DESC_st {
CUaddress_mode addressMode[3];
CUfilter_mode filterMode;
unsigned int flags;
unsigned int maxAnisotropy;
CUfilter_mode mipmapFilterMode;
float mipmapLevelBias;
float minMipmapLevelClamp;
float maxMipmapLevelClamp;
float borderColor[4];
int reserved[12];
} CUDA_TEXTURE_DESC;
/* Unused type */
typedef struct CUDA_RESOURCE_VIEW_DESC_st CUDA_RESOURCE_VIEW_DESC;
typedef unsigned int GLenum;
typedef unsigned int GLuint;
/*
* Prefix type name to avoid collisions. Clients using these types
* will include the real headers with real definitions.
*/
typedef int32_t ffnv_EGLint;
typedef void *ffnv_EGLStreamKHR;
typedef enum CUGLDeviceList_enum {
CU_GL_DEVICE_LIST_ALL = 1,
CU_GL_DEVICE_LIST_CURRENT_FRAME = 2,
CU_GL_DEVICE_LIST_NEXT_FRAME = 3,
} CUGLDeviceList;
typedef struct CUDA_EXTERNAL_MEMORY_HANDLE_DESC_st {
CUexternalMemoryHandleType type;
union {
int fd;
struct {
void *handle;
const void *name;
} win32;
} handle;
unsigned long long size;
unsigned int flags;
unsigned int reserved[16];
} CUDA_EXTERNAL_MEMORY_HANDLE_DESC;
typedef struct CUDA_EXTERNAL_MEMORY_BUFFER_DESC_st {
unsigned long long offset;
unsigned long long size;
unsigned int flags;
unsigned int reserved[16];
} CUDA_EXTERNAL_MEMORY_BUFFER_DESC;
typedef struct CUDA_EXTERNAL_SEMAPHORE_HANDLE_DESC_st {
CUexternalSemaphoreHandleType type;
union {
int fd;
struct {
void *handle;
const void *name;
} win32;
} handle;
unsigned int flags;
unsigned int reserved[16];
} CUDA_EXTERNAL_SEMAPHORE_HANDLE_DESC;
typedef struct CUDA_EXTERNAL_SEMAPHORE_SIGNAL_PARAMS_st {
struct {
struct {
unsigned long long value;
} fence;
unsigned int reserved[16];
} params;
unsigned int flags;
unsigned int reserved[16];
} CUDA_EXTERNAL_SEMAPHORE_SIGNAL_PARAMS;
typedef CUDA_EXTERNAL_SEMAPHORE_SIGNAL_PARAMS CUDA_EXTERNAL_SEMAPHORE_WAIT_PARAMS;
typedef struct CUDA_ARRAY_DESCRIPTOR_st {
size_t Width;
size_t Height;
CUarray_format Format;
unsigned int NumChannels;
} CUDA_ARRAY_DESCRIPTOR;
typedef struct CUDA_ARRAY3D_DESCRIPTOR_st {
size_t Width;
size_t Height;
size_t Depth;
CUarray_format Format;
unsigned int NumChannels;
unsigned int Flags;
} CUDA_ARRAY3D_DESCRIPTOR;
typedef struct CUDA_EXTERNAL_MEMORY_MIPMAPPED_ARRAY_DESC_st {
unsigned long long offset;
CUDA_ARRAY3D_DESCRIPTOR arrayDesc;
unsigned int numLevels;
unsigned int reserved[16];
} CUDA_EXTERNAL_MEMORY_MIPMAPPED_ARRAY_DESC;
#define CU_EGL_FRAME_MAX_PLANES 3
typedef struct CUeglFrame_st {
union {
CUarray pArray[CU_EGL_FRAME_MAX_PLANES];
void* pPitch[CU_EGL_FRAME_MAX_PLANES];
} frame;
unsigned int width;
unsigned int height;
unsigned int depth;
unsigned int pitch;
unsigned int planeCount;
unsigned int numChannels;
CUeglFrameType frameType;
CUeglColorFormat eglColorFormat;
CUarray_format cuFormat;
} CUeglFrame;
#define CU_STREAM_DEFAULT 0
#define CU_STREAM_NON_BLOCKING 1
#define CU_EVENT_DEFAULT 0
#define CU_EVENT_BLOCKING_SYNC 1
#define CU_EVENT_DISABLE_TIMING 2
#define CU_EVENT_WAIT_DEFAULT 0
#define CU_EVENT_WAIT_EXTERNAL 1
#define CU_TRSF_READ_AS_INTEGER 1
typedef void CUDAAPI CUstreamCallback(CUstream hStream, CUresult status, void *userdata);
typedef CUresult CUDAAPI tcuInit(unsigned int Flags);
typedef CUresult CUDAAPI tcuDriverGetVersion(int *driverVersion);
typedef CUresult CUDAAPI tcuDeviceGetCount(int *count);
typedef CUresult CUDAAPI tcuDeviceGet(CUdevice *device, int ordinal);
typedef CUresult CUDAAPI tcuDeviceGetAttribute(int *pi, CUdevice_attribute attrib, CUdevice dev);
typedef CUresult CUDAAPI tcuDeviceGetName(char *name, int len, CUdevice dev);
typedef CUresult CUDAAPI tcuDeviceGetUuid(CUuuid *uuid, CUdevice dev);
typedef CUresult CUDAAPI tcuDeviceGetUuid_v2(CUuuid *uuid, CUdevice dev);
typedef CUresult CUDAAPI tcuDeviceGetLuid(char* luid, unsigned int* deviceNodeMask, CUdevice dev);
typedef CUresult CUDAAPI tcuDeviceGetByPCIBusId(CUdevice* dev, const char* pciBusId);
typedef CUresult CUDAAPI tcuDeviceGetPCIBusId(char* pciBusId, int len, CUdevice dev);
typedef CUresult CUDAAPI tcuDeviceComputeCapability(int *major, int *minor, CUdevice dev);
typedef CUresult CUDAAPI tcuCtxCreate_v2(CUcontext *pctx, unsigned int flags, CUdevice dev);
typedef CUresult CUDAAPI tcuCtxGetCurrent(CUcontext *pctx);
typedef CUresult CUDAAPI tcuCtxSetLimit(CUlimit limit, size_t value);
typedef CUresult CUDAAPI tcuCtxPushCurrent_v2(CUcontext pctx);
typedef CUresult CUDAAPI tcuCtxPopCurrent_v2(CUcontext *pctx);
typedef CUresult CUDAAPI tcuCtxDestroy_v2(CUcontext ctx);
typedef CUresult CUDAAPI tcuMemAlloc_v2(CUdeviceptr *dptr, size_t bytesize);
typedef CUresult CUDAAPI tcuMemAllocPitch_v2(CUdeviceptr *dptr, size_t *pPitch, size_t WidthInBytes, size_t Height, unsigned int ElementSizeBytes);
typedef CUresult CUDAAPI tcuMemAllocManaged(CUdeviceptr *dptr, size_t bytesize, unsigned int flags);
typedef CUresult CUDAAPI tcuMemsetD8Async(CUdeviceptr dstDevice, unsigned char uc, size_t N, CUstream hStream);
typedef CUresult CUDAAPI tcuMemFree_v2(CUdeviceptr dptr);
typedef CUresult CUDAAPI tcuMemcpy(CUdeviceptr dst, CUdeviceptr src, size_t bytesize);
typedef CUresult CUDAAPI tcuMemcpyAsync(CUdeviceptr dst, CUdeviceptr src, size_t bytesize, CUstream hStream);
typedef CUresult CUDAAPI tcuMemcpy2D_v2(const CUDA_MEMCPY2D *pcopy);
typedef CUresult CUDAAPI tcuMemcpy2DAsync_v2(const CUDA_MEMCPY2D *pcopy, CUstream hStream);
typedef CUresult CUDAAPI tcuMemcpyHtoD_v2(CUdeviceptr dstDevice, const void *srcHost, size_t ByteCount);
typedef CUresult CUDAAPI tcuMemcpyHtoDAsync_v2(CUdeviceptr dstDevice, const void *srcHost, size_t ByteCount, CUstream hStream);
typedef CUresult CUDAAPI tcuMemcpyDtoH_v2(void *dstHost, CUdeviceptr srcDevice, size_t ByteCount);
typedef CUresult CUDAAPI tcuMemcpyDtoHAsync_v2(void *dstHost, CUdeviceptr srcDevice, size_t ByteCount, CUstream hStream);
typedef CUresult CUDAAPI tcuMemcpyDtoD_v2(CUdeviceptr dstDevice, CUdeviceptr srcDevice, size_t ByteCount);
typedef CUresult CUDAAPI tcuMemcpyDtoDAsync_v2(CUdeviceptr dstDevice, CUdeviceptr srcDevice, size_t ByteCount, CUstream hStream);
typedef CUresult CUDAAPI tcuGetErrorName(CUresult error, const char** pstr);
typedef CUresult CUDAAPI tcuGetErrorString(CUresult error, const char** pstr);
typedef CUresult CUDAAPI tcuCtxGetDevice(CUdevice *device);
typedef CUresult CUDAAPI tcuDevicePrimaryCtxRetain(CUcontext *pctx, CUdevice dev);
typedef CUresult CUDAAPI tcuDevicePrimaryCtxRelease(CUdevice dev);
typedef CUresult CUDAAPI tcuDevicePrimaryCtxSetFlags(CUdevice dev, unsigned int flags);
typedef CUresult CUDAAPI tcuDevicePrimaryCtxGetState(CUdevice dev, unsigned int *flags, int *active);
typedef CUresult CUDAAPI tcuDevicePrimaryCtxReset(CUdevice dev);
typedef CUresult CUDAAPI tcuStreamCreate(CUstream *phStream, unsigned int flags);
typedef CUresult CUDAAPI tcuStreamQuery(CUstream hStream);
typedef CUresult CUDAAPI tcuStreamSynchronize(CUstream hStream);
typedef CUresult CUDAAPI tcuStreamDestroy_v2(CUstream hStream);
typedef CUresult CUDAAPI tcuStreamAddCallback(CUstream hStream, CUstreamCallback *callback, void *userdata, unsigned int flags);
typedef CUresult CUDAAPI tcuStreamWaitEvent(CUstream hStream, CUevent hEvent, unsigned int flags);
typedef CUresult CUDAAPI tcuEventCreate(CUevent *phEvent, unsigned int flags);
typedef CUresult CUDAAPI tcuEventDestroy_v2(CUevent hEvent);
typedef CUresult CUDAAPI tcuEventSynchronize(CUevent hEvent);
typedef CUresult CUDAAPI tcuEventQuery(CUevent hEvent);
typedef CUresult CUDAAPI tcuEventRecord(CUevent hEvent, CUstream hStream);
typedef CUresult CUDAAPI tcuLaunchKernel(CUfunction f, unsigned int gridDimX, unsigned int gridDimY, unsigned int gridDimZ, unsigned int blockDimX, unsigned int blockDimY, unsigned int blockDimZ, unsigned int sharedMemBytes, CUstream hStream, void** kernelParams, void** extra);
typedef CUresult CUDAAPI tcuLinkCreate(unsigned int numOptions, CUjit_option* options, void** optionValues, CUlinkState* stateOut);
typedef CUresult CUDAAPI tcuLinkAddData(CUlinkState state, CUjitInputType type, void* data, size_t size, const char* name, unsigned int numOptions, CUjit_option* options, void** optionValues);
typedef CUresult CUDAAPI tcuLinkComplete(CUlinkState state, void** cubinOut, size_t* sizeOut);
typedef CUresult CUDAAPI tcuLinkDestroy(CUlinkState state);
typedef CUresult CUDAAPI tcuModuleLoadData(CUmodule* module, const void* image);
typedef CUresult CUDAAPI tcuModuleUnload(CUmodule hmod);
typedef CUresult CUDAAPI tcuModuleGetFunction(CUfunction* hfunc, CUmodule hmod, const char* name);
typedef CUresult CUDAAPI tcuModuleGetGlobal(CUdeviceptr *dptr, size_t *bytes, CUmodule hmod, const char* name);
typedef CUresult CUDAAPI tcuTexObjectCreate(CUtexObject* pTexObject, const CUDA_RESOURCE_DESC* pResDesc, const CUDA_TEXTURE_DESC* pTexDesc, const CUDA_RESOURCE_VIEW_DESC* pResViewDesc);
typedef CUresult CUDAAPI tcuTexObjectDestroy(CUtexObject texObject);
typedef CUresult CUDAAPI tcuGLGetDevices_v2(unsigned int* pCudaDeviceCount, CUdevice* pCudaDevices, unsigned int cudaDeviceCount, CUGLDeviceList deviceList);
typedef CUresult CUDAAPI tcuGraphicsGLRegisterImage(CUgraphicsResource* pCudaResource, GLuint image, GLenum target, unsigned int Flags);
typedef CUresult CUDAAPI tcuGraphicsUnregisterResource(CUgraphicsResource resource);
typedef CUresult CUDAAPI tcuGraphicsMapResources(unsigned int count, CUgraphicsResource* resources, CUstream hStream);
typedef CUresult CUDAAPI tcuGraphicsUnmapResources(unsigned int count, CUgraphicsResource* resources, CUstream hStream);
typedef CUresult CUDAAPI tcuGraphicsSubResourceGetMappedArray(CUarray* pArray, CUgraphicsResource resource, unsigned int arrayIndex, unsigned int mipLevel);
typedef CUresult CUDAAPI tcuGraphicsResourceGetMappedPointer(CUdeviceptr *devPtrOut, size_t *sizeOut, CUgraphicsResource resource);
typedef CUresult CUDAAPI tcuImportExternalMemory(CUexternalMemory* extMem_out, const CUDA_EXTERNAL_MEMORY_HANDLE_DESC* memHandleDesc);
typedef CUresult CUDAAPI tcuDestroyExternalMemory(CUexternalMemory extMem);
typedef CUresult CUDAAPI tcuExternalMemoryGetMappedBuffer(CUdeviceptr* devPtr, CUexternalMemory extMem, const CUDA_EXTERNAL_MEMORY_BUFFER_DESC* bufferDesc);
typedef CUresult CUDAAPI tcuExternalMemoryGetMappedMipmappedArray(CUmipmappedArray* mipmap, CUexternalMemory extMem, const CUDA_EXTERNAL_MEMORY_MIPMAPPED_ARRAY_DESC* mipmapDesc);
typedef CUresult CUDAAPI tcuMipmappedArrayGetLevel(CUarray* pLevelArray, CUmipmappedArray hMipmappedArray, unsigned int level);
typedef CUresult CUDAAPI tcuMipmappedArrayDestroy(CUmipmappedArray hMipmappedArray);
typedef CUresult CUDAAPI tcuImportExternalSemaphore(CUexternalSemaphore* extSem_out, const CUDA_EXTERNAL_SEMAPHORE_HANDLE_DESC* semHandleDesc);
typedef CUresult CUDAAPI tcuDestroyExternalSemaphore(CUexternalSemaphore extSem);
typedef CUresult CUDAAPI tcuSignalExternalSemaphoresAsync(const CUexternalSemaphore* extSemArray, const CUDA_EXTERNAL_SEMAPHORE_SIGNAL_PARAMS* paramsArray, unsigned int numExtSems, CUstream stream);
typedef CUresult CUDAAPI tcuWaitExternalSemaphoresAsync(const CUexternalSemaphore* extSemArray, const CUDA_EXTERNAL_SEMAPHORE_WAIT_PARAMS* paramsArray, unsigned int numExtSems, CUstream stream);
typedef CUresult CUDAAPI tcuArrayCreate(CUarray *pHandle, const CUDA_ARRAY_DESCRIPTOR* pAllocateArray);
typedef CUresult CUDAAPI tcuArray3DCreate(CUarray *pHandle, const CUDA_ARRAY3D_DESCRIPTOR* pAllocateArray);
typedef CUresult CUDAAPI tcuArrayDestroy(CUarray hArray);
typedef CUresult CUDAAPI tcuEGLStreamProducerConnect(CUeglStreamConnection* conn, ffnv_EGLStreamKHR stream, ffnv_EGLint width, ffnv_EGLint height);
typedef CUresult CUDAAPI tcuEGLStreamProducerDisconnect(CUeglStreamConnection* conn);
typedef CUresult CUDAAPI tcuEGLStreamConsumerDisconnect(CUeglStreamConnection* conn);
typedef CUresult CUDAAPI tcuEGLStreamProducerPresentFrame(CUeglStreamConnection* conn, CUeglFrame eglframe, CUstream* pStream);
typedef CUresult CUDAAPI tcuEGLStreamProducerReturnFrame(CUeglStreamConnection* conn, CUeglFrame* eglframe, CUstream* pStream);
typedef CUresult CUDAAPI tcuD3D11GetDevice(CUdevice *device, void *dxgiAdapter);
typedef CUresult CUDAAPI tcuD3D11GetDevices(unsigned int *deviceCountOut, CUdevice *devices, unsigned int deviceCount, void *d3d11device, CUd3d11DeviceList listType);
typedef CUresult CUDAAPI tcuGraphicsD3D11RegisterResource(CUgraphicsResource *cudaResourceOut, void *d3d11Resource, unsigned int flags);
#endif
File diff suppressed because it is too large. Load diff