The probe always used Auto (CRF), so the ABR path of the bitrate presets
(#15) could not be measured without a viewer.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Review follow-ups to f0506bc: a stalled encoder read no longer shortens the
next capture interval, and the capture process exit status is logged
because nothing awaits its task.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The root geometry and XShm image were read once at startup, so after a
RandR or xrdp resize every new connection asked x11grab for the old size
and stalled on "transferring picture". Query the root geometry before
each desktop connection and rebuild the capture when it changed; guard
capture with a lock and reject a stale-sized buffer instead of
overflowing it. The HEVC fallback log now includes the actual failure.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The native Wayland session relayed each 33 MB 4K frame through this process
and the capture loop wrote synchronously, so a slow encoder read delayed the
next screencopy request (~65 fps at a 144 fps request versus 87-92 fps for
bare capture, #16). Hand the capture stdout to FFmpeg directly, double-buffer
capture against the pipe write on a separate thread, and enlarge the pipe.
Terminal: hold the size lock across the ioctl so the delayed redraw restore
cannot reapply a stale size, and record scrollback and live output under the
sink lock so a reattach never replays a chunk that is also sent live.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
sent/skipped are cumulative while max_* are per-window peaks, which was
easy to misread (#16). Add window_s, window_sent and window_skipped.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Reattaching replays the scrollback tail, which starts mid-redraw for
full-screen programs (tmux, vim, TUIs), and an unchanged size raises no
SIGWINCH. Shrink the PTY by one row and restore the latest requested size
after a pause so the program repaints (#17).
Apply the #15 ABR change to the libx265 path as well, clarify the
non-wayland build error, and document the Wayland systemd unit and uinput
module autoload (#16).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* Wayland: keep capturing when output power management is unavailable
keep_display_awake() runs on the desktop video start path, so a compositor\nwithout wlr-output-power-management made uu_wl_wake() return -1 and aborted\nthe whole desktop stream. Treat the missing protocol as nothing to wake.
* x264: average bitrate for the bitrate presets so 4K keyframes fit
Capped CRF plus VBV cannot fit a large IDR frame at 4K and a low cap, so\nkeyframes came out near-grey. NVENC and VAAPI already carry a target bitrate;\ngive x264 the same and keep CRF only for automatic quality.
One video track carries H.264 or H.265 (RFC 7798 payloader); the host
announces its encoders with CONTROL Message 18 CodecNegotiation and picks
H.265 when the viewer negotiated it (the official Mac viewer offers H265
first; CaptureSettingRequest field 13 only reports the current codec).
hevc_nvenc / libx265 / hevc_vaapi with fallback to software HEVC, then
H.264; UU_VIDEO_CODEC=h264 forces H.264. True color 4:4:4 applies to H.265.
Verified with the official Mac viewer on NVENC.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The viewer sends CaptureSettingRequest field 7 = 3 when 真彩 is enabled
(ff_capture_setting versions are cumulative, so our version 4 already
implies chroma sampling). Encode yuv444p with NVENC high444p / x264
high444 and report the encoded chroma in CaptureSettingSync; VAAPI and
inproc stay 4:2:0. Log ConnectOptions decoder_cap_list (numbers only).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Squashed from the Codex v0.1.9 branch (reviewed):
- Clear stale input when a desktop session starts or ends and at service
start/stop: X11 releases every pressed key/button on the display (a stuck
XTest button blocked all clicks after a VM reboot), uinput resets its own
devices, Wayland/portal sweep the keys this host can emit. A per-desktop
lease stops late callbacks of an old connection from touching a new one;
file/terminal/port-mapping peers never release desktop input.
- Native Wayland (--features wayland): double-buffered screencopy pipeline,
viewer frame rate drives the capture rate, opt-in dma-buf (UU_WAYLAND_DMABUF=1,
linear only, falls back to SHM). VM-measured only.
- Internal QA records (docs/validation, contrib/tests) removed from the public
tree and ignored; they stay in the private project.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The 500 kbit cap left a 3296x2036 IDR (~130 KB at the 2M preset) no room, so
x264 emitted near-grey keyframes that sharpened over each 0.5 s GOP. Reproduced
with a static 3296x2036 desktop-like image at the 2M preset over 6 s: 250k and
1000k buffers produce grey keyframes (154/360 frames at 1000k), 2000k and 4000k
none. NVENC ull and VAAPI keep the small low-latency buffer. Thanks to
@liangjiahui01 for the analysis.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
ffmpeg lists h264_nvenc even without NVIDIA hardware, so AMD/Intel hosts ran a
doomed NVENC process before every libx264 stream. UU_VIDEO_ENCODER=nvenc forces it.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The in-process encoder can now capture through XShm and XDamage in pure Rust
(x11rb) instead of libav's x11grab. Damage marks the picture dirty; a dirty
picture is captured and encoded on a fixed frame tick so the stream keeps a
steady cadence. An idle screen sends a keepalive every 40-67 ms (15-25 fps,
random) so the viewer does not treat a static desktop as stalled.
- src/damage_capture.rs: XDamage NonEmpty reports and XShm copy-out
- src/inproc_encoder.rs: run_damage loop; x11grab stays as the fallback
- UU_CAPTURE=grab forces the x11grab path
- x11rb is an optional dependency, enabled by the inproc-av feature only
Measured on a static desktop, CPU drops from about 12% to about 1% versus
x11grab. Moving content costs about the same CPU as x11grab.
Address PR review:
- The FFmpeg 6 bindings (AVChannelLayout, ffmpeg-next 6.1) do not build on
the 22.04 runners, which ship FFmpeg 4.4. The default build no longer
depends on libav at all, so `cargo test --locked` and the release job on
22.04 no longer compile that code.
- The release tarball no longer links the build machine's libav SONAMEs.
Default builds use the FFmpeg child process for video and audio again.
- `UU_FFMPEG_ROOT` / config `ffmpeg_root` now selects the child-process path
even in an inproc-av build, so a bundled FFmpeg with working NVENC is not
silently bypassed.
Build with `--features inproc-av` (FFmpeg 6 dev packages, Ubuntu 24.04) to
get in-process capture, encoding and Opus. CI has a test-inproc job for it.
`UU_VIDEO_PIPELINE=process` still forces the child path for video.
Probes on ros2go-gpu (440 Hz tone, 5 s):
default (ffmpeg child): 44.0 packets/s, 132 kbps, max gap 43 ms
inproc-av: 49.7 packets/s, 133.5 kbps, max gap 22 ms
desktop-video-probe under inproc-av with UU_FFMPEG_ROOT=/usr uses the child
path (no `pipeline=in-process` in the log); without it, in-process.
Static desktop, desktop-video-probe, 600 frames, NVENC CQ18, same run:
GOP 0.5 s (30 frames) 7740 kbps 20 IDR
GOP 2 s (120 frames) 1908 kbps 5 IDR
A broken prediction chain now freezes until the next IDR or a PLI. The
PLI/FIR path from the previous commit makes the recovery prompt when the
viewer sends one. It has not been checked against a viewer that loses
packets, so a lossy link may freeze for up to 2 s. UU_VIDEO_GOP_SECONDS
overrides the interval for tuning.
Capture (x11grab) and H.264 encoding (h264_nvenc / libx264) now run in
this process through ffmpeg-next, with the same codec options as the old
ffmpeg CLI. Each encoder packet is one access unit, fed to the existing
DependencyGate. A PLI or FIR from the viewer forces the next frame to be
an IDR. Wayland and portal capture still use the child-process path.
Set UU_VIDEO_PIPELINE=process to fall back to the old ffmpeg child.
desktop-video-probe accepts UU_PROBE_FRAMES to lengthen the sample.
Measured with desktop-video-probe on ros2go-gpu (RTX 4090 D, Xvfb
1920x1080 @60fps, 600 frames, mostly static desktop). CPU is the sum of
the probe process and its children, sampled from /proc once per second.
Probe only: no WebRTC send, no audio.
encoder pipeline CPU (%) wall (s) bitrate (kbps)
nvenc process ~138 10.17 5356
nvenc in-process ~43 10.34 5176
libx264 process ~135 10.09 5073
libx264 in-process ~72 10.08 4986
All four outputs decode without errors and start with an IDR.
Not covered: audio still runs its own ffmpeg child (src/audio.rs), PLI to
IDR has not been checked against a live viewer, and the CI jobs need the
libav*-dev and libclang-dev packages (added to ci.yml, release.yml,
AGENTS.md and the READMEs).
Unofficial Linux host for NetEase UU Remote: X11 (production-tested on
GNOME/Ubuntu 24.04), experimental Wayland (Hyprland) and portal
(GNOME/KDE) backends; video, input, clipboard, file transfer, terminal,
sound, quality/fps/resolution settings, lock/unlock and system actions.
Co-Authored-By: Claude Opus 5.5 <[email protected]>