Compare commits

...
2 Commits
13 changed files with 487 additions and 31 deletions

No files matched your search

Generated
+1 -1
View File
@@ -2741,7 +2741,7 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uu-remote-linux"
version = "0.2.13"
version = "0.2.14"
dependencies = [
"async-trait",
"base64 0.22.1",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "uu-remote-linux"
version = "0.2.13"
version = "0.2.14"
edition = "2021"
default-run = "uuremote-server"
authors = ["uu-remote-linux contributors"]
+12 -2
View File
@@ -80,7 +80,7 @@ The GNOME Wayland Portal/EIS path has been checked for video and mouse/keyboard
real hardware, please [open an issue](../../issues) with your distribution, desktop, GPU and the relevant lines from
`journalctl --user -u uuremote-native.service`.
Wayland limits: single screen, fixed resolution. The native Hyprland backend now pipelines two shm screencopy buffers and passes the viewer's requested fps to both capture pacing and the encoder. Actual fps remains limited by the compositor, rendering and readback (the old serial path measured about 16–22 fps at 2560x1600; the pipeline is VM-measured only, not a hardware guarantee). The portal backend does not support dynamic resolution; see below for GNOME wake and lock recovery; the clipboard syncs while the viewer's remote window has focus. On the first connection the desktop asks for "Remote Desktop / Screen Sharing"
Wayland limits: captures one screen by default; Hyprland can enable monitor switching as described below. Resolution is fixed. The native Hyprland backend now pipelines two shm screencopy buffers and passes the viewer's requested fps to both capture pacing and the encoder. Actual fps remains limited by the compositor, rendering and readback (the old serial path measured about 16–22 fps at 2560x1600; the pipeline is VM-measured only, not a hardware guarantee). The portal backend does not support dynamic resolution; see below for GNOME wake and lock recovery; the clipboard syncs while the viewer's remote window has focus. On the first connection the desktop asks for "Remote Desktop / Screen Sharing"
permission; the permission is remembered once granted.
Scaled GNOME desktops using native Portal input use EIS on the same authorized session when `ConnectToEIS` and a screen `mapping_id` are available. Each axis is mapped using the matched region and actual frame dimensions, with no fixed resolution or scale factor. This avoids scaled-coordinate rejection by `NotifyPointerMotionAbsolute`. Portal builds require libei >= 1.1. Missing EIS support retains the legacy input path and its existing scaling limitations; other initialization failures are reported and trigger session recovery. After a successful EIS connection, all input uses EIS. Capture alone and uinput do not initialize EIS; missing optional logical dimensions do not block capture. Keyboard input uses the existing Windows VK to evdev mapping. Recovery keeps the encoder frame dimensions fixed but accepts logical-size changes after scaling. On GNOME, a display-change notification enables earlier recovery when previously streaming Portal capture remains paused without a valid format for a 200 ms grace after both conditions are present. Healthy renegotiation cancels the early check; repeated notifications do not postpone the same interruption. The five-second fallback remains without a related notification. Static video and valid-format pauses do not trigger recovery. Tested while connected on Ubuntu 26.04, GNOME Wayland, one 3840x2160 display: 200% -> 166.67% -> 200%, with continuously updating video and full-screen pointer control after recovery. Switching briefly freezes and goes black before recovering; five seconds is the fallback threshold, not total recovery time. See [#35](../../pull/35) for timing measurements and validation limits. Lock recovery still needs hardware acceptance testing; multi-monitor support, live resolution changes, and rotation/reflection transforms are outside this fix's scope. Capture settings remain at protocol version 4: advertised resolutions use pixel coordinates and DPR 1; Portal logical dimensions are used only for local input mapping.
@@ -103,11 +103,19 @@ failure falls back to SHM. The virgl VM verified fallback only: successful impor
mapping and hardware acceleration remain unverified. This is not zero-copy VAAPI;
`UU_VIDEO_ENCODER=vaapi` still uploads CPU frames through FFmpeg.
### Hyprland monitor switching (experimental)
Native Wayland absolute pointer motion now targets the captured output by default, fixing multi-monitor click offsets. uinput may still handle keyboard, buttons and relative motion.
Set `UU_WAYLAND_MULTISCREEN=1` to select physical monitors from the phone client. `UU_WAYLAND_OUTPUT` chooses the initial output. Capture and absolute pointer targeting switch together across different resolutions, scales and desktop positions.
Switching reuses one video track and pauses input until the new source sends a frame. Capture/encoder failure or a first-frame timeout requests restoration of the previous source. The inventory uses `hyprctl` and supports unrotated outputs; restart the host after monitor hotplug or layout changes. Phone-side switching and clicks were tested on Omarchy / Hyprland with 2560×1440 at 125% and 1920×1080 at 133.33%.
## Not implemented / not supported
| Feature | Status |
|---|---|
| Switching between multiple monitors | In progress |
| Switching between multiple monitors | Experimental on native Hyprland Wayland; see above |
| True color (4:4:4) | Supported (v0.1.15): the default FFmpeg pipeline with NVENC / libx264 encodes H.264 High 4:4:4; VAAPI and `inproc-av` stay 4:2:0 and report that to the client |
| HDR | Not implemented |
| Super screen / virtual display | Not implemented |
@@ -301,6 +309,8 @@ Common environment variables:
|---|---|
| `UU_CONFIG` | Config file path (default `~/.config/uuremote/config.json`) |
| `UU_DESKTOP_BACKEND` | `auto` / `x11` / `wayland` / `portal` / `kms` (experimental feature) / `none` (headless) |
| `UU_WAYLAND_MULTISCREEN=1` | Enable Hyprland physical monitor switching (off by default) |
| `UU_WAYLAND_OUTPUT` | Initial native Wayland output name, e.g. `HDMI-A-1` |
| `UU_INPUT_BACKEND` | `auto` (default: uinput on Wayland when accessible) / `uinput` / `native` |
| `UU_TERMINAL_SHORTCUTS=0` | Disable the Ctrl+V paste rewrite in terminal windows |
| `UU_VIDEO_ENCODER` | `cpu`: libx264; `vaapi`: opt-in Intel/AMD; `nvenc`: force NVENC; unset: NVENC only when an NVIDIA driver is loaded, else CPU |
+12 -2
View File
@@ -73,7 +73,7 @@ Portal 仅作用于本服务获授权的会话。文件传输、终端等独立
GNOME Wayland 的 Portal/EIS 路径已在 Ubuntu 26.04、GNOME 50、单屏 4K/200% 缩放下验证画面和键鼠;锁屏冷连接自动解锁已用 Mac 和 iPhone 4.42.0 客户端实机验证,其余锁屏恢复及重连仍待验收。除 GNOME 的上述路径和 X11 外,其余桌面目前主要在虚拟机里测试。如果你在真机上遇到问题,欢迎[提交 issue](../../issues),
请附上发行版、桌面环境、显卡型号和 `journalctl --user -u uuremote-native.service` 里的相关日志。
Wayland 下的限制:只支持单屏、固定分辨率;Hyprland 原生后端默认使用双缓冲流水线 shm screencopy,控制端帧率设置会同步到采集节拍和编码器,但实际帧率仍受合成器、渲染和读回速度限制(旧串行路径在 2560x1600 曾测得约 16~22 帧;流水线只在虚拟机里测量过,不代表实机保证);portal 后端暂不支持动态分辨率;GNOME 的屏幕唤醒与锁屏恢复见下;剪贴板只在控制端的远控窗口处于前台时同步。
Wayland 下的限制:默认采集单屏,Hyprland 可按上文启用切屏;分辨率固定;Hyprland 原生后端默认使用双缓冲流水线 shm screencopy,控制端帧率设置会同步到采集节拍和编码器,但实际帧率仍受合成器、渲染和读回速度限制(旧串行路径在 2560x1600 曾测得约 16~22 帧;流水线只在虚拟机里测量过,不代表实机保证);portal 后端暂不支持动态分辨率;GNOME 的屏幕唤醒与锁屏恢复见下;剪贴板只在控制端的远控窗口处于前台时同步。
GNOME 缩放桌面在使用 Portal 原生键鼠时,若支持 `ConnectToEIS` 和屏幕 `mapping_id`,通过同一授权会话的 EIS 接口注入输入,按匹配的区域与实际帧尺寸分别映射横纵坐标,没有固定分辨率或缩放倍率,可避免 `NotifyPointerMotionAbsolute` 在缩放时拒绝坐标。Portal 构建需要 libei ≥ 1.1。缺少 EIS 支持时保留旧输入路径及其既有缩放限制;其他初始化错误会明确报告并触发会话恢复。EIS 连接成功后全部输入统一使用 EIS。只采集画面或使用 uinput 时不会初始化 EIS;缺少可选的逻辑尺寸也不阻断采集。键盘沿用项目现有的 Windows VK → evdev 映射。恢复时保持编码器的采集帧尺寸不变,允许缩放后的逻辑尺寸变化。GNOME 显示配置变化后,已推流的 Portal 采集若持续「暂停且无有效格式」,会在关联通知后的 200 毫秒宽限结束后提前进入现有恢复流程;正常重协商恢复会取消提前判定,连续通知不会推迟同次异常的判定。未收到相关通知时保留五秒兜底;静态画面或有效格式下的暂停不会触发恢复。Ubuntu 26.04、GNOME Wayland 单屏 3840×2160 已实测保持连接切换 200%→166.67%→200%,恢复后画面持续更新且鼠标可控制全屏;切换期间会短暂停顿、黑屏后恢复,五秒是兜底判定阈值,并非总恢复时间。提前恢复的测量结果和验证范围见 [#35](../../pull/35)。此路径仍需实机验证锁屏恢复;多屏、连接中改变分辨率及旋转/镜像变换不属于本次修复范围。 捕获设置协议仍为版本 4:对客户端上报的分辨率统一使用像素坐标、DPR 为 1;Portal 逻辑尺寸仅用于本地输入映射。
@@ -90,11 +90,19 @@ GNOME Portal 的控制通道不再等待授权窗口或第一帧画面:先通
原生 Wayland 实验选项:`UU_WAYLAND_PIPELINE=0` 可关闭流水线做对照。`UU_WAYLAND_DMABUF=1` 显式尝试 dma-buf→GBM CPU 映射(默认关闭,需要运行时 libgbm;可用 `UU_WAYLAND_DMABUF_DEVICE` 指定 render node,默认 `/dev/dri/renderD128`)。仅支持显式线性修饰符、单平面 8-bit RGB,失败回退 SHM;virgl VM 中只验证了回退,成功导入/映射和硬件加速均未验证。这不是零拷贝 VAAPI 路径;选择 `UU_VIDEO_ENCODER=vaapi` 时仍由 FFmpeg 上传 CPU 帧。
首次连接时,桌面会弹窗请求「远程桌面 / 屏幕共享」授权,同意后会记住授权。
### Hyprland 多显示器切换(实验性)
原生 Wayland 的绝对鼠标定位默认绑定正在采集的输出,修复双屏桌面下的点击偏移。uinput 可继续用于键盘、按键和相对移动。
设置 `UU_WAYLAND_MULTISCREEN=1` 后,手机端可选择并切换物理显示器;`UU_WAYLAND_OUTPUT` 指定默认屏幕。画面与绝对鼠标定位同步切换,支持不同分辨率、缩放和桌面位置。
切屏复用同一条视频轨道,过渡期间暂停输入,发送新屏首帧后恢复;采集或编码失败、等待首帧超时会请求恢复原屏。当前通过 `hyprctl` 枚举未旋转的显示器,显示器插拔或布局变化后需重启服务。已在 Omarchy / Hyprland 上用官方手机客户端实测 2560×1440(125%)与 1920×1080(133.33%)双屏切换及点击。
## 尚未实现 / 不支持
| 功能 | 状态 |
|---|---|
| 多显示器切换 | 开发中 |
| 多显示器切换 | Hyprland 原生 Wayland 后端实验性支持,见上文 |
| 真彩(4:4:4) | 支持(v0.1.15):NVENC / libx264 的默认 FFmpeg 管线输出 H.264 High 4:4:4;VAAPI 和 `inproc-av` 管线仍为 4:2:0,并如实回报给客户端 |
| HDR | 未实现 |
| 超级屏 / 虚拟显示器 | 未实现 |
@@ -272,6 +280,8 @@ systemctl --user restart uuremote-native.service
|---|---|
| `UU_CONFIG` | 配置文件路径(默认 `~/.config/uuremote/config.json`) |
| `UU_DESKTOP_BACKEND` | `auto` / `x11` / `wayland` / `portal` / `kms`(实验性特性) / `none`(无头模式) |
| `UU_WAYLAND_MULTISCREEN=1` | 启用 Hyprland 物理显示器切换(默认关闭) |
| `UU_WAYLAND_OUTPUT` | 原生 Wayland 默认采集输出名称,例如 `HDMI-A-1` |
| `UU_INPUT_BACKEND` | `auto`(默认,Wayland 下可用时用 uinput)/ `uinput` / `native` |
| `UU_TERMINAL_SHORTCUTS=0` | 关闭终端窗口里的 Ctrl+V 粘贴转换 |
| `UU_VIDEO_ENCODER` | `cpu` 强制 CPU;`vaapi` 显式启用 Intel/AMD 硬编;`nvenc` 强制 NVENC;未设置时检测到 NVIDIA 驱动才用 NVENC,否则用 CPU |
+27
View File
@@ -571,6 +571,33 @@ end:
if(result!=0) reset();
pthread_mutex_unlock(&lock); return result;
}
/* Switch the pointer binding and main-process capture atomically; encoder subprocesses
select their own output via UU_WAYLAND_OUTPUT. Keyboard identity stays unchanged. */
int uu_wl_select_output(const char *name,uint32_t expected_w,uint32_t expected_h) {
pthread_mutex_lock(&lock); int result=-1;
if(!initialized || !name || sync_display()!=0) goto end;
unsigned target;
for(target=0;target<output_count;target++) if(!strcmp(outputs[target].name,name)) break;
if(target==output_count || outputs[target].transform!=WL_OUTPUT_TRANSFORM_NORMAL) goto end;
uint32_t w=outputs[target].width,h=outputs[target].height;
if(w!=expected_w || h!=expected_h || !w || !h || w>16384 || h>16384 || (uint64_t)w*h*4>256*1024*1024) goto end;
struct zwlr_virtual_pointer_v1 *next=zwlr_virtual_pointer_manager_v1_create_virtual_pointer_with_output(pointer_manager,seat,outputs[target].handle);
if(!next) goto end;
if(sync_display()!=0) {zwlr_virtual_pointer_v1_destroy(next);goto end;}
for(unsigned i=0;i<2;i++) { frame_clear(&frames[i]); dma_clear(&frames[i]); shm_clear(&frames[i]); }
pending=0;
if(pointer) zwlr_virtual_pointer_v1_destroy(pointer);
pointer=next; selected=target; width=w; height=h;
if(power) { zwlr_output_power_v1_destroy(power); power=NULL; }
power_failed=0;
if(power_manager) {
power=zwlr_output_power_manager_v1_get_output_power(power_manager,outputs[selected].handle);
zwlr_output_power_v1_add_listener(power,&power_listener,NULL);
}
result=0;
end:
pthread_mutex_unlock(&lock); return result;
}
int uu_wl_wake(void) {
pthread_mutex_lock(&lock); int result=-1;
/* Output power management is optional; missing it must not block screencopy. */
+1 -1
View File
@@ -1,7 +1,7 @@
# Unofficial UU Remote Linux host in a container, for NAS boxes (Synology DSM, etc.) without a desktop.
# Headless mode: file transfer, terminal and port mapping; no remote desktop. Uses the published release binary.
FROM ubuntu:24.04
ARG VERSION=v0.2.13
ARG VERSION=v0.2.14
LABEL org.opencontainers.image.source="https://github.com/VallierDev/uu-remote-linux" \
org.opencontainers.image.description="Unofficial UU Remote Linux host, headless (file transfer, terminal, port mapping)" \
org.opencontainers.image.licenses="GPL-3.0"
+20
View File
@@ -0,0 +1,20 @@
Hyprland 双屏点击偏移修复,并可在手机端切换显示器(实验性)。/ Hyprland dual-monitor pointer fix and opt-in monitor switching.
### 修复 / Fixed
- **Hyprland 双屏点击偏移**(#57,感谢 @JuanZoran):原生 Wayland 后端的绝对鼠标定位改为绑定正在采集的输出
(此前 uinput 绝对坐标映射到整个桌面,双屏时点击位置偏移)。uinput 仍用于键盘、按键和相对移动。
**影响所有 Hyprland 用户的默认路径**;单屏下映射结果与之前等价。贡献者已在双屏真机(2560×1440@125% + 1920×1080@133%)
上验证落点误差 ≤ 2 像素;维护者未另行复测。
- Absolute pointer motion on the native Wayland backend is now bound to the captured output (it previously spanned the whole
desktop through uinput, offsetting clicks on multi-monitor setups). Verified by the contributor on real dual-monitor hardware.
### 新增 / Added
- **Hyprland 多显示器切换(实验性,默认关闭)**:设置 `UU_WAYLAND_MULTISCREEN=1` 后,手机端可以选择物理显示器,画面与
鼠标定位同步切换;`UU_WAYLAND_OUTPUT` 指定默认屏幕。通过 `hyprctl` 枚举未旋转的显示器,插拔或布局变化后需重启服务;
切换失败会恢复原屏。由贡献者用官方手机客户端在 Omarchy / Hyprland 双屏上实测。
- Opt-in Hyprland monitor switching (`UU_WAYLAND_MULTISCREEN=1`), accepted by the contributor with the official phone client.
### 下载 / Download
`uu-remote-linux-v0.2.14-x86_64-linux.tar.gz`(含 `libuu-wayland.so` / `libuu-portal.so`),或 `docker pull ghcr.io/vallierdev/uu-remote-linux:v0.2.14`。
**非官方项目,与网易无关。** / **Unofficial, not affiliated with NetEase.**
+51 -2
View File
@@ -40,6 +40,8 @@ impl VideoCodec {
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct VideoSettings {
pub source_id: usize,
pub source_generation: u64,
pub fps: u32,
pub codec: VideoCodec,
pub quality: u64,
@@ -52,7 +54,7 @@ pub struct VideoSettings {
}
impl Default for VideoSettings {
fn default() -> Self { Self { codec: VideoCodec::H264, fps: 60, quality: QUALITY_AUTO, custom_mbps: 0, output: None, chroma: CHROMA_420 } }
fn default() -> Self { let (source_id,source_generation)=crate::wayland::selection(); Self { source_id,source_generation, codec: VideoCodec::H264, fps: 60, quality: QUALITY_AUTO, custom_mbps: 0, output: None, chroma: CHROMA_420 } }
}
/// Selectable smaller stream sizes with the desktop's aspect ratio (even dimensions).
@@ -180,9 +182,37 @@ fn apply_rpc(rpc: &[WireField], settings: &watch::Sender<VideoSettings>, native:
Some(out)
}
/// CaptureSettingResponse.errors (Error fields 1 code / 2 message, rpc_main.proto).
pub fn rejected_request(data:&[u8],reason:&str)->Option<Vec<u8>> {
let outer=decode_wire(data).ok()?;
let rpc=decode_wire(message(&outer,21)?).ok()?;
message(&rpc,2)?;
let mut error=Vec::new();
put_varint(1,(-1i64) as u64,&mut error);
put_bytes(2,reason.as_bytes(),&mut error);
let mut errors=Vec::new();put_bytes(1,&error,&mut errors);
let mut response=Vec::new();
put_bytes(1,message(&rpc,1).unwrap_or(&[]),&mut response);
put_bytes(2,&errors,&mut response);
Some(response)
}
/// Message 7 ScreenSources in pixels for capture-setting v4: native, choices, current size.
pub fn screen_sources(seq: u64, timestamp: u64, native: (u32, u32), output: Option<(u32, u32)>, name: &str) -> Result<Vec<u8>, String> {
use crate::initialization::{Screen, WinRect};
let outputs=crate::wayland::outputs();
if !outputs.is_empty() {
let current=crate::wayland::selected();
let choices:Vec<Vec<(i32,i32)>>=outputs.iter().map(|m|scaled_sizes((m.width,m.height)).into_iter().map(|(w,h)|(w as i32,h as i32)).collect()).collect();
let screens:Vec<Screen<'_>>=outputs.iter().enumerate().map(|(id,m)| Screen {
id:id as i32, fps:m.fps,
resolution:WinRect {left:0,top:0,width:m.width as i32,height:m.height as i32,pixel_width:m.width as i32,pixel_height:m.height as i32},
primary:id==0,dpr:1.0,display_name:&m.name,video_track_index:if id==current {0} else {-1},
screen_type:0,builtin_screen_type:if m.name.starts_with("eDP") {1} else {2},scaled:&choices[id],
current:if id==current {output.map(|(w,h)|(w as i32,h as i32))} else {None},
}).collect();
return crate::initialization::encode_screen_sources(seq,timestamp,&screens,current as i32);
}
let (w, h) = (native.0 as i32, native.1 as i32);
let scaled: Vec<(i32, i32)> = scaled_sizes(native).into_iter().map(|(w, h)| (w as i32, h as i32)).collect();
// DPI_SCALE starts at version 5; the observed v4 request uses fields 5/6 as pixels.
@@ -248,7 +278,7 @@ mod tests {
put_varint(18, 30, &mut body);
let request = rpc(77, 2, &body);
let response = handle_rpc(&decode_wire(&request).unwrap(), &tx, (1920, 1080)).unwrap();
assert_eq!(*rx.borrow(), VideoSettings { codec: VideoCodec::H264, fps: 30, quality: QUALITY_GENERAL, custom_mbps: 0, output: None, chroma: CHROMA_420 });
assert_eq!(*rx.borrow(), VideoSettings { source_id:0,source_generation:0, codec: VideoCodec::H264, fps: 30, quality: QUALITY_GENERAL, custom_mbps: 0, output: None, chroma: CHROMA_420 });
assert_eq!(rx.borrow().max_kbps(), Some(2_000));
let fields = decode_wire(&response).unwrap();
assert_eq!(message(&fields, 1), Some(&[8u8, 77][..]));
@@ -470,3 +500,22 @@ mod codec_tests {
assert_eq!(preferred_codec(true, false), VideoCodec::H264);
}
}
#[cfg(test)]
mod screen_error_tests {
use super::*;
#[test]
fn failed_switch_returns_correlated_error_not_success() {
let header=[10,4,b't',b'e',b's',b't'];
let mut rpc=Vec::new();put_bytes(1,&header,&mut rpc);put_bytes(2,&[32,99],&mut rpc);
let request=crate::initialization::envelope(1,2,21,&rpc);
let response=rejected_request(&request,"Unknown screen").unwrap();
let fields=decode_wire(&response).unwrap();
assert_eq!(message(&fields,1),Some(header.as_slice()));
let result=decode_wire(message(&fields,2).unwrap()).unwrap();
let error=decode_wire(message(&result,1).unwrap()).unwrap();
assert_eq!(number(&error,1),Some(u64::MAX));
assert_eq!(message(&error,2),Some(b"Unknown screen".as_slice()));
assert!(rejected_request(b"invalid","failure").is_none());
}
}
+14 -3
View File
@@ -77,8 +77,8 @@ pub struct Screen<'a> {
}
impl Screen<'_> {
fn encode(&self) -> Result<Vec<u8>, String> {
if self.id < 0 || self.fps <= 0 || self.video_track_index < 0 {
return Err("Screen id/track must be nonnegative and fps positive".into());
if self.id < 0 || self.fps <= 0 || self.video_track_index < -1 {
return Err("Screen id must be nonnegative, track >= -1 and fps positive".into());
}
if !self.dpr.is_finite() || self.dpr <= 0.0 {
return Err("Screen dpr must be positive and finite".into());
@@ -118,7 +118,7 @@ pub fn encode_screen_sources(seq: u64, timestamp: u64, screens: &[Screen<'_>], c
let mut tracks = std::collections::HashSet::new();
let mut payload = Vec::new();
for screen in screens {
if !ids.insert(screen.id) || !tracks.insert(screen.video_track_index) {
if !ids.insert(screen.id) || (screen.video_track_index >= 0 && !tracks.insert(screen.video_track_index)) {
return Err("Screen ids and video track indices must be unique".into());
}
bytes(1, &screen.encode()?, &mut payload);
@@ -295,3 +295,14 @@ mod tests {
assert!(encode_capture_change(1, 2, 4, 1, "").is_err());
}
}
#[cfg(test)]
mod inactive_track_tests {
use super::*;
#[test]
fn multiple_inactive_screens_share_minus_one_but_active_track_is_unique() {
let screen=|id,track| Screen {id,fps:60,resolution:WinRect {left:0,top:0,width:1920,height:1080,pixel_width:1920,pixel_height:1080},primary:id==0,dpr:1.0,display_name:"test",video_track_index:track,screen_type:0,builtin_screen_type:0,scaled:&[],current:None};
assert!(encode_screen_sources(1,2,&[screen(0,0),screen(1,-1),screen(2,-1)],0).is_ok());
assert!(encode_screen_sources(1,2,&[screen(0,0),screen(1,0)],0).is_err());
}
}
+58
View File
@@ -325,6 +325,12 @@ impl InputInjector {
}
}
fn inject_move(&self,x:i32,y:i32,relative:bool)->i32 {
// uinput ABS axes span the compositor's whole desktop. A captured output
// uses its own pixel space; bind absolute motion to that Wayland output,
// including when uinput remains in use for keyboard/buttons/relative input.
if !relative && self.backend==crate::desktop_backend::DesktopBackend::Wayland {
return crate::wayland::move_pointer(x,y,false);
}
if let Some(uinput)=&self.uinput {
if relative { return Self::uinput_result(uinput.move_relative(x,y)); }
let (w,h)=*self.screen.lock().unwrap();
@@ -639,3 +645,55 @@ mod lifecycle_tests {
session.with_input(||panic!("late event after disconnect"));
}
}
#[cfg(all(test,target_os="linux",feature="wayland"))]
mod output_pointer_tests {
use super::*;
#[test]
#[ignore="requires unlocked Hyprland, UU_WAYLAND_MULTISCREEN=1 and accessible uinput; moves pointer only"]
fn absolute_pointer_stays_on_capture_output_with_uinput_enabled() {
fn cursor()->serde_json::Value {
let output=std::process::Command::new("hyprctl").args(["cursorpos","-j"]).output().unwrap();
assert!(output.status.success());serde_json::from_slice(&output.stdout).unwrap()
}
struct Restore(serde_json::Value);
impl Drop for Restore {
fn drop(&mut self) {
let _=std::process::Command::new("hyprctl").args(["dispatch","movecursor",&self.0["x"].to_string(),&self.0["y"].to_string()]).output();
}
}
let _restore=Restore(cursor());
crate::wayland::init().unwrap();
let input=InputInjector::new();
assert!(input.uinput.is_some(),"test must exercise the uinput/native hybrid path");
// libinput discovers the newly created virtual devices asynchronously.
std::thread::sleep(std::time::Duration::from_secs(1));
let output=std::process::Command::new("hyprctl").args(["monitors","-j"]).output().unwrap();
let monitors:Vec<serde_json::Value>=serde_json::from_slice(&output.stdout).unwrap();
let outputs=crate::wayland::outputs();assert!(outputs.len()>=2,"needs at least two outputs");
for (id,output) in outputs.iter().enumerate().chain(outputs.iter().enumerate().rev()) {
crate::wayland::select(id).unwrap();
let monitor=monitors.iter().find(|m|m["name"].as_str()==Some(output.name.as_str())).unwrap();
let scale=monitor["scale"].as_f64().unwrap();
for (x,y) in [(0.1,0.1),(0.5,0.5),(0.9,0.9),(0.1,0.9),(0.9,0.1)] {
let size=(output.width,output.height);
let pixels=crate::input_mapping::normalized_pointer(x,y,size.0,size.1).unwrap();
assert_eq!(input.backend,crate::desktop_backend::DesktopBackend::Wayland);
input.mouse_absolute_normalized(x,y,size);input.flush_pointer();
let expected=(monitor["x"].as_f64().unwrap()+f64::from(pixels.0)/scale,
monitor["y"].as_f64().unwrap()+f64::from(pixels.1)/scale);
// Motion is asynchronous; observe its first arrival instead of a
// later position potentially changed by another input device.
let deadline=std::time::Instant::now()+std::time::Duration::from_millis(250);
loop {
let actual=cursor();
let matched=(actual["x"].as_f64().unwrap()-expected.0).abs()<=2.0
&& (actual["y"].as_f64().unwrap()-expected.1).abs()<=2.0;
if matched {break;}
assert!(std::time::Instant::now()<deadline,"pointer offset: output={} normalized=({},{}) actual={} expected={:?}",output.name,x,y,actual,expected);
std::thread::sleep(std::time::Duration::from_millis(5));
}
}
}
}
}
+52 -9
View File
@@ -160,6 +160,8 @@ fn is_independent_frame(bytes: &[u8]) -> bool { is_independent_frame_for(bytes,
#[derive(Debug)]
struct CompleteFrame {
source_id: usize,
source_generation: u64,
codec: VideoCodec,
sequence: u64,
completed_at: Instant,
@@ -167,6 +169,12 @@ struct CompleteFrame {
_permit: Option<OwnedSemaphorePermit>,
}
impl CompleteFrame {
fn matches(&self,settings:&VideoSettings)->bool {
self.codec==settings.codec && self.source_id==settings.source_id && self.source_generation==settings.source_generation
}
}
const RTP_CLOCK: u64 = 90_000;
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
@@ -527,7 +535,7 @@ async fn read_inprocess(
let mut gate = DependencyGate::default();
while let Some(unit) = units.recv().await {
let sequence = sequence.fetch_add(1, std::sync::atomic::Ordering::Relaxed) + 1;
gate.submit(&latest, Arc::new(CompleteFrame { codec, sequence, completed_at: unit.completed_at, data: unit.data, _permit: Some(unit.permit) }));
gate.submit(&latest, Arc::new(CompleteFrame { source_generation: 0, source_id: 0, codec, sequence, completed_at: unit.completed_at, data: unit.data, _permit: Some(unit.permit) }));
if latest.is_closed() { break; }
}
// Dropping the receiver also unblocks the encoder if it is waiting to send.
@@ -574,6 +582,8 @@ async fn read_encoder_with(
size: (u32,u32),display:String,portal:Option<Arc<crate::portal::PortalSession>>,latest:mpsc::Sender<Arc<CompleteFrame>>,encoder:Encoder,sequence:Arc<std::sync::atomic::AtomicU64>,
settings: VideoSettings, #[cfg_attr(not(feature = "inproc-av"), allow(unused_variables))] idr: Arc<AtomicBool>, budget: Arc<Semaphore>,
)->Result<(),String>{
let selected_output=if display.starts_with("wayland:") {crate::wayland::source(settings.source_id)} else {None};
let size=selected_output.map_or(size,|s|(s.width,s.height));
let kms=display.starts_with("kms:");
let wayland=kms || display.starts_with("wayland:");
let (permits_tx,mut permits_rx)=mpsc::channel::<OwnedSemaphorePermit>(4);
@@ -637,7 +647,9 @@ async fn read_encoder_with(
let executable=std::path::PathBuf::from("/proc/self/exe");
#[cfg(not(target_os="linux"))]
let executable=std::env::current_exe().map_err(|e|format!("Capture executable: {e}"))?;
let mut capture=wayland_capture_command(&executable,settings.fps)
let mut capture_command=wayland_capture_command(&executable,settings.fps);
if let Some(output)=selected_output {capture_command.env("UU_WAYLAND_OUTPUT",&output.name);}
let mut capture=capture_command
.env("UU_WAYLAND_DEMAND","1")
.kill_on_drop(true).stdin(std::process::Stdio::piped()).stdout(std::process::Stdio::piped()).stderr(std::process::Stdio::inherit())
.spawn().map_err(|e|format!("Start Wayland capture: {e}"))?;
@@ -735,7 +747,7 @@ async fn read_encoder_with(
let frame: Vec<u8> = pending.drain(..next).collect();
let sequence = sequence.fetch_add(1, std::sync::atomic::Ordering::Relaxed) + 1;
let permit=if wayland {Some(permits_rx.recv().await.ok_or("Capture producer stopped")?)} else {None};
gate.submit(&latest, Arc::new(CompleteFrame {
gate.submit(&latest, Arc::new(CompleteFrame { source_generation: settings.source_generation, source_id: settings.source_id,
codec: settings.codec, sequence, completed_at: Instant::now(), data: frame, _permit: permit,
}));
if latest.is_closed() { return Ok(()); }
@@ -921,7 +933,7 @@ pub async fn stream_desktop(pc: Arc<RTCPeerConnection>, track: Arc<crate::video_
'stream: loop {
let next = *settings.borrow_and_update();
if next != current || send_interrupted {
let discard = next.codec != current.codec || send_interrupted;
let discard = next.codec != current.codec || next.source_generation != current.source_generation || next.source_id != current.source_id || send_interrupted;
current = next;
timeline.fps = u64::from(next.fps);
if discard {
@@ -945,6 +957,19 @@ pub async fn stream_desktop(pc: Arc<RTCPeerConnection>, track: Arc<crate::video_
reader.0 = tokio::spawn(read_encoder(size, display.clone(), portal.clone(), latest.clone(), current, sequence.clone(), idr.clone(), budget.clone()));
continue;
}
if result.is_err() && !crate::wayland::outputs().is_empty() && !crate::wayland::input_ready() {
// Leave the stream task alive so the request worker can roll back
// a failed capture/encoder and restart the previous source.
crate::wayland::failed(current.source_id,current.source_generation);
loop {
tokio::select! {
changed=settings.changed()=>{if changed.is_err(){return result;}break;},
_=health.tick()=>{if transport_closed(&pc){return Ok(());}},
}
}
send_interrupted=true;
continue;
}
return result;
},
_ = health.tick() => { if transport_closed(&pc) { return Ok(()); } },
@@ -955,7 +980,7 @@ pub async fn stream_desktop(pc: Arc<RTCPeerConnection>, track: Arc<crate::video_
frame = receiver.recv() => {
// A closed queue means the reader ended; its JoinHandle carries the reason.
let Some(frame) = frame else { return (&mut reader.0).await.map_err(|e| format!("Encoder reader task: {e}"))?; };
if frame.codec != current.codec { continue; }
if !frame.matches(&current) { continue; }
let duration = timeline.duration_after(frame.completed_at);
if let Some(last) = previous { skipped += frame.sequence.saturating_sub(last + 1); }
max_age_ms = max_age_ms.max(frame.completed_at.elapsed().as_millis());
@@ -964,6 +989,7 @@ pub async fn stream_desktop(pc: Arc<RTCPeerConnection>, track: Arc<crate::video_
data: frame.data.clone().into(), duration, ..Default::default()
};
let extensions = [crate::rtp_metadata::capture_index(0), crate::rtp_metadata::new_frame()];
let sent_before=track.revision();
let send = track.write_sample_with_extensions(frame.codec, &sample, &extensions);
let send_started=Instant::now();
tokio::pin!(send);
@@ -977,6 +1003,9 @@ pub async fn stream_desktop(pc: Arc<RTCPeerConnection>, track: Arc<crate::video_
},
}
}
if !crate::wayland::outputs().is_empty() && track.revision()>sent_before {
crate::wayland::presented(current.source_id,current.source_generation);
}
previous = Some(frame.sequence);
max_send_ms=max_send_ms.max(send_started.elapsed().as_millis());
frames += 1;
@@ -1082,7 +1111,7 @@ mod tests {
let mut gate=DependencyGate::default();
for sequence in 1..=2 {
let permit=budget.clone().acquire_owned().await.unwrap();
gate.submit(&tx,Arc::new(CompleteFrame {codec:VideoCodec::H264,sequence,completed_at:Instant::now(),
gate.submit(&tx,Arc::new(CompleteFrame { source_generation: 0, source_id: 0,codec:VideoCodec::H264,sequence,completed_at:Instant::now(),
data:vec![0,0,1,5,42],_permit:Some(permit)}));
}
let sending=rx.recv().await.unwrap();
@@ -1216,7 +1245,7 @@ mod tests {
let (tx, mut rx) = mpsc::channel(4);
let old_tx = tx.clone();
let (started_tx, started_rx) = tokio::sync::oneshot::channel();
let frame = |sequence, idr| Arc::new(CompleteFrame { _permit: None,
let frame = |sequence, idr| Arc::new(CompleteFrame { source_generation: 0, source_id: 0, _permit: None,
sequence, codec: VideoCodec::H264, completed_at: Instant::now(),
data: vec![0, 0, 1, 9, 240, 0, 0, 1, if idr { 5 } else { 1 }, 42],
});
@@ -1311,7 +1340,7 @@ mod tests {
async fn bounded_queue_never_sends_broken_prediction_chain_after_overload(){
let(tx,mut rx)=mpsc::channel(1);
let mut gate=DependencyGate::default();
let frame=|sequence,idr|Arc::new(CompleteFrame{_permit:None,sequence,codec:VideoCodec::H264,completed_at:Instant::now(),data:vec![0,0,1,9,240,0,0,1,if idr{5}else{1},42]});
let frame=|sequence,idr|Arc::new(CompleteFrame{source_generation:0,source_id:0,_permit:None,sequence,codec:VideoCodec::H264,completed_at:Instant::now(),data:vec![0,0,1,9,240,0,0,1,if idr{5}else{1},42]});
gate.submit(&tx,frame(1,false));assert!(rx.try_recv().is_err()); // no IDR yet
gate.submit(&tx,frame(2,true));
gate.submit(&tx,frame(3,false)); // queue full: P3 lost
@@ -1365,7 +1394,7 @@ mod tests {
async fn hevc_dependency_gate_recovers_only_at_irap_after_overflow() {
let (tx, mut rx) = mpsc::channel(1);
let mut gate = DependencyGate::default();
let frame = |sequence, irap| Arc::new(CompleteFrame { _permit: None, codec: VideoCodec::H265,
let frame = |sequence, irap| Arc::new(CompleteFrame { source_generation: 0, source_id: 0, _permit: None, codec: VideoCodec::H265,
sequence, completed_at: Instant::now(), data: vec![0,0,1,if irap { 38 } else { 2 },1,42] });
gate.submit(&tx, frame(1, false));
assert!(rx.try_recv().is_err());
@@ -1407,3 +1436,17 @@ mod tests {
}
}
#[cfg(test)]
mod screen_frame_tests {
use super::*;
#[test]
fn source_restart_rejects_queued_frames_even_with_same_screen_and_codec() {
let frame=CompleteFrame {source_id:0,source_generation:7,codec:VideoCodec::H264,sequence:1,completed_at:Instant::now(),data:vec![],_permit:None};
let mut settings=VideoSettings {source_id:0,source_generation:7,..Default::default()};
assert!(frame.matches(&settings));
settings.source_generation=9;assert!(!frame.matches(&settings));
settings.source_generation=7;settings.source_id=1;assert!(!frame.matches(&settings));
settings.source_id=0;settings.codec=VideoCodec::H265;assert!(!frame.matches(&settings));
}
}
+159
View File
@@ -1,6 +1,7 @@
//! Hyprland/wlroots native backend. No XWayland fallback and no input logging.
#[cfg(all(target_os="linux", feature="wayland"))]
crate::engine_ffi! { crate::engine_plugin::Engine::Wayland;
fn uu_wl_select_output(name: *const libc::c_char, width: u32, height: u32) -> i32;
fn uu_wl_init(out_w: *mut i32, out_h: *mut i32) -> i32;
fn uu_wl_capture(buffer: *mut u8, length: usize) -> i32;
fn uu_wl_move(x: i32, y: i32, relative: i32) -> i32;
@@ -44,6 +45,13 @@ pub fn init() -> Result<(u32,u32),String> {
if result != 0 || w <= 0 || h <= 0 {
return Err(format!("Wayland initialization failed ({result}); requires screencopy, virtual keyboard and virtual pointer. Check UU_WAYLAND_OUTPUT and session environment."));
}
// The compositor registry and hyprctl need not enumerate outputs in the
// same order. Bind the initial engine to the inventory's selected output.
static BINDING: std::sync::OnceLock<Result<(),String>>=std::sync::OnceLock::new();
if !outputs().is_empty() {
BINDING.get_or_init(||select(0)).as_ref().map_err(Clone::clone)?;
return Ok(input_size((w as u32,h as u32)));
}
return Ok((w as u32,h as u32));
}
#[cfg(not(all(target_os="linux", feature="wayland")))]
@@ -195,3 +203,154 @@ pub fn text_probe()->Result<(),String> {
sequence.chars().filter(|character|*character!='|').count(),sequence.split('|').count());
Ok(())
}
// Opt-in single-track monitor switching for Hyprland. The existing native engine owns
// capture/pointer binding; no compositor layout or user credentials are modified.
#[derive(Clone, Debug)]
pub struct Output { pub name:String, pub width:u32, pub height:u32, pub fps:i32 }
static OUTPUTS: std::sync::OnceLock<Vec<Output>> = std::sync::OnceLock::new();
#[derive(Default)]
struct Selection { id:usize, generation:u64, presented:Option<u64>, failed:bool }
impl Selection {
fn begin(&mut self,id:usize) {
self.id=id; self.generation+=1; self.presented=None; self.failed=false;
}
fn sent(&mut self,id:usize,generation:u64) {
if (self.id,self.generation)==(id,generation) {self.presented=Some(generation);}
}
fn ready(&self)->bool {self.presented==Some(self.generation)}
}
static SELECTION:std::sync::Mutex<Selection>=std::sync::Mutex::new(Selection {id:0,generation:0,presented:None,failed:false});
pub fn outputs() -> &'static [Output] {
OUTPUTS.get_or_init(|| {
if !cfg!(all(target_os="linux",feature="wayland"))
|| std::env::var("UU_WAYLAND_MULTISCREEN").ok().as_deref()!=Some("1")
|| crate::desktop_backend::detect().ok()!=Some(crate::desktop_backend::DesktopBackend::Wayland) {return Vec::new();}
let output=std::process::Command::new("hyprctl").args(["monitors","-j"]).output();
let result=output.map_err(|_|"hyprctl unavailable").and_then(|output| {
if !output.status.success() {return Err("hyprctl failed");}
parse_outputs(&output.stdout,&std::env::var("UU_WAYLAND_OUTPUT").unwrap_or_default())
});
match result {Ok(outputs)=>outputs,Err(reason)=>{log::warn!("Wayland multi-screen disabled: {}",reason);Vec::new()}}
})
}
fn parse_outputs(json:&[u8],preferred:&str)->Result<Vec<Output>,&'static str> {
let monitors=serde_json::from_slice::<Vec<serde_json::Value>>(json).map_err(|_|"Invalid monitor inventory")?;
let mut seen=std::collections::HashSet::new();
let mut outputs:Vec<Output>=monitors.into_iter().filter_map(|m| {
if m["disabled"].as_bool()==Some(true) || m["transform"].as_u64()!=Some(0) {return None;}
let name=m["name"].as_str()?;
let width=u32::try_from(m["width"].as_u64()?).ok()?;
let height=u32::try_from(m["height"].as_u64()?).ok()?;
if name.is_empty() || name.len()>127 || name.contains('\0') || !seen.insert(name.to_owned())
|| width<2 || height<2 || width>16384 || height>16384 || u64::from(width)*u64::from(height)*4>256*1024*1024 {return None;}
Some(Output {name:name.into(),width,height,fps:m["refreshRate"].as_f64()?.round().clamp(1.0,240.0) as i32})
}).collect();
outputs.sort_by_key(|m| (m.name!=preferred,m.name.clone()));
outputs.truncate(16);
if outputs.is_empty() || (!preferred.is_empty() && outputs[0].name!=preferred) {return Err("No capturable preferred output");}
Ok(outputs)
}
pub fn selection()->(usize,u64) {let s=SELECTION.lock().unwrap();(s.id,s.generation)}
pub fn selected()->usize {selection().0}
pub fn source(id:usize)->Option<&'static Output> {outputs().get(id)}
pub fn input_size(fallback:(u32,u32))->(u32,u32) {source(selected()).map_or(fallback,|s|(s.width,s.height))}
pub fn input_ready()->bool {outputs().is_empty() || SELECTION.lock().unwrap().ready()}
pub fn switch_failed()->bool {SELECTION.lock().unwrap().failed}
pub fn presented(id:usize,generation:u64) {SELECTION.lock().unwrap().sent(id,generation);}
pub fn failed(id:usize,generation:u64) {
let mut state=SELECTION.lock().unwrap();
if (state.id,state.generation)==(id,generation) {state.failed=true;}
}
pub fn select(id:usize)->Result<(),String> {
let output=source(id).ok_or("Unknown Wayland screen")?;
#[cfg(all(target_os="linux",feature="wayland"))]
{
let name=std::ffi::CString::new(output.name.as_str()).map_err(|_|"Invalid output name")?;
let mut state=SELECTION.lock().unwrap();
if unsafe {uu_wl_select_output(name.as_ptr(),output.width,output.height)}!=0 {return Err("Wayland output switch failed".into());}
state.begin(id);
log::info!("Wayland screen selected: id={} source={}x{}",id,output.width,output.height);
Ok(())
}
#[cfg(not(all(target_os="linux",feature="wayland")))]
{let _=output;Err("Wayland output switching unavailable".into())}
}
/// CaptureSettingRequest.screen_id is field 4 (rpc_main.proto); official desktop
/// activation also uses SimpleAction 8 {"screen_id":id}. Reference: openuuyc main
/// a87b8e7, stream_control/wire.rs and publisher/displays.rs (Windows S456070).
/// https://github.com/djkcyl/openuuyc/tree/a87b8e772c238e542215ab039f1b1d2ca4a9ef46/src/features/stream_control
/// Official phone acceptance on Hyprland: field 4 selected each advertised id;
/// current-source settings used the signed value -1.
pub fn requested_output(data:&[u8]) -> Option<usize> {
use crate::protocol::{decode_wire,WireValue};
let outer=decode_wire(data).ok()?;
fn bytes<'a>(fields:&[crate::protocol::WireField<'a>],n:u32)->Option<&'a [u8]> {fields.iter().find_map(|f|match f.value {WireValue::Bytes(b) if f.number==n=>Some(b),_=>None})}
if let Some(rpc)=bytes(&outer,21) {
let rpc=decode_wire(rpc).ok()?;
let request=decode_wire(bytes(&rpc,2)?).ok()?;
let id=request.iter().find_map(|f|match f.value {WireValue::Varint(v) if f.number==4=>Some(v as i64),_=>None}).unwrap_or(0);
return usize::try_from(id).ok();
}
let action=decode_wire(bytes(&outer,3)?).ok()?;
let kind=action.iter().find_map(|f|match f.value {WireValue::Varint(v) if f.number==1=>Some(v),_=>None})?;
if kind!=8 {return None;}
let args:serde_json::Value=serde_json::from_slice(bytes(&action,2)?).ok()?;
usize::try_from(args["screen_id"].as_i64()?).ok()
}
#[cfg(test)]
mod monitor_switch_tests {
use super::*;
#[test]
fn capture_setting_screen_zero_and_signed_current_target() {
// Message.rpc_request(21), RpcRequest.capture_setting(2), screen_id(4).
let request=|payload:&[u8]| {let mut rpc=vec![18,payload.len() as u8];rpc.extend(payload);crate::initialization::envelope(1,2,21,&rpc)};
assert_eq!(requested_output(&request(&[32,1])),Some(1));
assert_eq!(requested_output(&request(&[])),Some(0));
assert_eq!(requested_output(&request(&[32,255,255,255,255,255,255,255,255,255,1])),None);
assert_eq!(requested_output(b"not protobuf"),None);
}
#[test]
fn only_explicit_desktop_restart_selects_an_output() {
let request=|action:u8,args:&str| {let mut sa=vec![8,action,18,args.len() as u8];sa.extend(args.bytes());crate::initialization::envelope(1,2,3,&sa)};
assert_eq!(requested_output(&request(8,r#"{"screen_id":1}"#)),Some(1));
assert_eq!(requested_output(&request(8,r#"{"screen_id":-1}"#)),None);
assert_eq!(requested_output(&request(7,r#"{"screen_id":1}"#)),None);
}
}
#[cfg(test)]
mod selection_regressions {
use super::*;
#[test]
fn old_frames_cannot_unlock_a_return_to_the_same_screen() {
let mut s=Selection::default();
s.begin(0);let old=s.generation;s.sent(0,old);assert!(s.ready());
s.begin(1);assert!(!s.ready());s.sent(0,old);assert!(!s.ready());
s.begin(0);let current=s.generation;
s.sent(0,old);assert!(!s.ready());
s.sent(1,current);assert!(!s.ready());
s.sent(0,current);assert!(s.ready());
}
#[test]
fn mixed_scale_inventory_preserves_pixel_sizes_and_preferred_output() {
let json=br#"[{"name":"eDP-1","width":1920,"height":1080,"scale":1.3333334,"transform":0,"refreshRate":144},
{"name":"HDMI-A-1","width":2560,"height":1440,"scale":1.25,"transform":0,"refreshRate":75}]"#;
let outputs=parse_outputs(json,"HDMI-A-1").unwrap();
assert_eq!(outputs.iter().map(|m|(m.name.as_str(),m.width,m.height)).collect::<Vec<_>>(),
vec![("HDMI-A-1",2560,1440),("eDP-1",1920,1080)]);
assert!(parse_outputs(json,"missing-output").is_err());
}
#[test]
fn unusable_outputs_are_not_advertised() {
let json=br#"[{"name":"rotated","width":1920,"height":1080,"transform":1,"refreshRate":60},
{"name":"too-large","width":16000,"height":16000,"transform":0,"refreshRate":60},
{"name":"off","width":1920,"height":1080,"disabled":true,"transform":0,"refreshRate":60},
{"name":"normal","width":1920,"height":1080,"transform":0,"refreshRate":60}]"#;
let outputs=parse_outputs(json,"").unwrap();
assert_eq!(outputs.len(),1);assert_eq!(outputs[0].name,"normal");
assert!(parse_outputs(b"not JSON","").is_err());
}
}
+79 -10
View File
@@ -391,11 +391,14 @@ impl WebRtcSession {
});
}
// Settings arrive on CONTROL or TEXT; serialize both queues for this peer.
let settings_serial=Arc::new(Mutex::new(()));
// 监听 DataChannel
pc.on_data_channel(Box::new(move |d: Arc<RTCDataChannel>| {
let label = d.label().to_string();
log::info!("New DataChannel received: {}", label);
let peer_settings_serial=settings_serial.clone();
let input_inst = input_clone.clone();
let channel_input_session=input_session.clone();
let screen = control_capture.clone();
@@ -445,7 +448,7 @@ impl WebRtcSession {
});
}
let (capture_tx, mut capture_rx) = tokio::sync::mpsc::channel::<Vec<u8>>(16);
if label == "CONTROL_DATA_CHANNEL" {
if matches!(label.as_str(),"CONTROL_DATA_CHANNEL"|"TEXT_DATA_CHANNEL") {
let settings = channel_video_settings.clone();
let track = peer_video_track.clone();
let response_tx = outbound.clone();
@@ -453,6 +456,8 @@ impl WebRtcSession {
let manager = portal_desktop.clone();
let channel = Arc::downgrade(&d);
let initial_screen = screen.clone();
let switch_owner=channel_input_session.clone();
let settings_serial=peer_settings_serial.clone();
tokio::spawn(async move {
let mut health = tokio::time::interval(std::time::Duration::from_millis(100));
loop {
@@ -467,6 +472,36 @@ impl WebRtcSession {
continue;
}
};
let _serial=settings_serial.lock().await;
if desktop && switch_owner.as_ref().is_none_or(|owner|!owner.is_current()) {continue;}
let reject=|reason:&str| {
if let Some(response)=crate::capture_setting::rejected_request(&request,reason) {
let ts=std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap_or_default().as_secs();
let _=response_tx.send(crate::initialization::envelope(sequence.fetch_add(1,std::sync::atomic::Ordering::Relaxed),ts,22,&response));
}
};
let source_before=crate::wayland::selected();
let settings_before=*settings.borrow();
if desktop && !crate::wayland::outputs().is_empty() {
if let Some(id)=crate::wayland::requested_output(&request) {
if crate::wayland::source(id).is_none() {reject("Unknown screen");continue;}
if id!=source_before {
let mut changed=false;
if let Some(owner)=&switch_owner {owner.with_current(|| {
owner.release_input();
match crate::wayland::select(id) {
Ok(())=>{
let (source_id,source_generation)=crate::wayland::selection();
settings.send_modify(|s|{s.source_id=source_id;s.source_generation=source_generation;s.output=None;});
changed=true;
},
Err(error)=>{log::warn!("Screen switch rejected: {}",error);reject(&error);},
}
});}
if !changed {continue;}
}
}
}
let active_screen = if desktop {
if let Some(manager) = &manager {
loop {
@@ -492,6 +527,33 @@ impl WebRtcSession {
let input_size = if active_screen.init_error.is_none() {
(active_screen.width, active_screen.height)
} else { (1920, 1080) };
let input_size=crate::wayland::input_size(input_size);
let source_changed=source_before!=crate::wayland::selected();
if source_changed {
let target=crate::wayland::selection();
let deadline=tokio::time::Instant::now()+std::time::Duration::from_secs(10);
while crate::wayland::selection()==target && !crate::wayland::input_ready()
&& !crate::wayland::switch_failed() && tokio::time::Instant::now()<deadline {
if channel.upgrade().is_none_or(|c|c.ready_state()!=webrtc::data_channel::data_channel_state::RTCDataChannelState::Open)
|| switch_owner.as_ref().is_none_or(|owner|!owner.is_current()) {return;}
tokio::time::sleep(std::time::Duration::from_millis(20)).await;
}
if switch_owner.as_ref().is_none_or(|owner|!owner.is_current()) || crate::wayland::selection()!=target {continue;}
if !crate::wayland::input_ready() {
if let Some(owner)=&switch_owner {owner.with_current(|| {
if crate::wayland::selection()==target && crate::wayland::select(source_before).is_ok() {
let (source_id,source_generation)=crate::wayland::selection();
settings.send_replace(crate::capture_setting::VideoSettings {source_id,source_generation,..settings_before});
}
});}
log::warn!("Screen switch failed; previous source requested");
reject("Screen switch failed; restoring previous screen");
continue;
}
let ts=std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap_or_default().as_secs();
if let Ok(message)=crate::capture_setting::screen_sources(sequence.fetch_add(1,std::sync::atomic::Ordering::Relaxed),ts,input_size,None,"Linux desktop") {let _=response_tx.send(message);}
if let Ok(message)=crate::initialization::encode_capture_change(sequence.fetch_add(1,std::sync::atomic::Ordering::Relaxed),ts,0,crate::wayland::selected() as i32,"Linux desktop") {let _=response_tx.send(message);}
}
let Ok(fields) = crate::protocol::decode_wire(&request) else { continue; };
use crate::protocol::WireValue;
let Some(rpc) = fields.iter().find_map(|f| match (f.number, &f.value) {
@@ -501,10 +563,14 @@ impl WebRtcSession {
let before = *settings.borrow();
let output_before = before.output;
let revision = track.revision();
let Some(response) = crate::capture_setting::handle_rpc_with_codecs(
let apply=|| crate::capture_setting::handle_rpc_with_codecs(
&rpc_fields, &settings, input_size,
track.supports(crate::capture_setting::VideoCodec::H264),
track.supports(crate::capture_setting::VideoCodec::H265)) else { continue; };
track.supports(crate::capture_setting::VideoCodec::H265));
let response=if let Some(owner)=&switch_owner {
let mut response=None;owner.with_current(||response=apply());response
} else {apply()};
let Some(response)=response else {continue;};
let codec_changed = settings.borrow().codec != before.codec;
if desktop && codec_changed && !track.wait_for_applied(revision, &settings).await {
log::warn!("Codec switch did not produce a transmitted frame; success reply withheld");
@@ -645,7 +711,7 @@ impl WebRtcSession {
.duration_since(std::time::UNIX_EPOCH).unwrap_or_default().as_secs();
let message = crate::initialization::encode_capture_change(
sequence.fetch_add(1, std::sync::atomic::Ordering::Relaxed),
timestamp, 0, 0, "Linux desktop").expect("desktop capture type");
timestamp, 0, crate::wayland::selected() as i32, "Linux desktop").expect("desktop capture type");
if let Err(error) = channel.send(&bytes::Bytes::from(message)).await {
log::error!("CONTROL desktop selection failed: {error}");
return;
@@ -733,9 +799,9 @@ impl WebRtcSession {
} else { crate::system_metrics::Permission::Reject };
// Without a desktop (headless/portal before consent) the port-mapping tool still waits
// for ScreenSources/CaptureChange and aborts otherwise: tool peers get a nominal screen.
let screen_size=if screen.init_error.is_none() { (screen.width, screen.height) } else { (1920, 1080) };
let screen_size=crate::wayland::input_size(if screen.init_error.is_none() { (screen.width, screen.height) } else { (1920, 1080) });
let messages = [
(8, crate::initialization::encode_capture_change(next_seq(), timestamp, 0, 0, "Linux desktop")),
(8, crate::initialization::encode_capture_change(next_seq(), timestamp, 0, crate::wayland::selected() as i32, "Linux desktop")),
(13, Ok(crate::initialization::envelope(next_seq(), timestamp, 13, &init_actions.metrics_payload(desktop_sound, desktop_permission)))),
(26, Ok(crate::initialization::envelope(next_seq(), timestamp, 26,
&crate::system_metrics::remote_download_path(&home)))),
@@ -822,11 +888,12 @@ impl WebRtcSession {
let capture_requests = capture_tx.clone();
Box::pin(async move {
let input_size = (screen.width, screen.height);
let input_size = crate::wayland::input_size((screen.width, screen.height));
if let Some(term) = &terminal_worker {
if crate::terminal::is_frame(&msg.data) { term.handle(&msg.data); return; }
}
if channel_name == "CONTROL_DATA_CHANNEL" && is_capture_setting_message(&msg.data) {
if matches!(channel_name.as_str(),"CONTROL_DATA_CHANNEL"|"TEXT_DATA_CHANNEL")
&& (is_capture_setting_message(&msg.data) || (!crate::wayland::outputs().is_empty() && crate::wayland::requested_output(&msg.data).is_some())) {
// Back-pressure like the old inline handler instead of dropping the latest setting.
if capture_requests.send(msg.data.to_vec()).await.is_err() {
log::warn!("Capture setting queue closed; request dropped");
@@ -1070,7 +1137,7 @@ impl WebRtcSession {
}
}
if let Ok(crate::control_protocol::ControlMessage::Input(crate::control_protocol::ControlInput::TextInput {text})) = crate::control_protocol::decode_control(&msg.data) {
if let Some(session)=message_input_session.as_ref() { session.with_input(|| input_worker.text_input(&text)); }
if let Some(session)=message_input_session.as_ref() { session.with_input(|| {if crate::wayland::input_ready() {input_worker.text_input(&text);}}); }
}
}
if channel_name.contains("CONTROL") {
@@ -1078,6 +1145,7 @@ impl WebRtcSession {
Ok(crate::control_protocol::ControlMessage::Input(event)) => {
if screen.backend == crate::desktop_backend::DesktopBackend::Portal
&& (screen.init_error.is_some() || !desktop_ready || *lock_for_message.borrow() != Some(false)) { return; }
if !crate::wayland::input_ready() {return;}
use crate::control_protocol::ControlInput;
// Separate receipt budgets keep motion floods from hiding clicks.
static RECEIPTS: [std::sync::atomic::AtomicUsize; 3] = [const { std::sync::atomic::AtomicUsize::new(0) }; 3];
@@ -1090,8 +1158,9 @@ impl WebRtcSession {
log::info!("CONTROL input received: kind={}", kind);
}
if let Some(session)=message_input_session.as_ref() {session.with_input(|| {
if !crate::wayland::input_ready() {return;}
match &event {
ControlInput::MouseAbsolute {x,y} => input_worker.mouse_absolute_normalized(*x,*y,input_size),
ControlInput::MouseAbsolute {x,y} => input_worker.mouse_absolute_normalized(*x,*y,crate::wayland::input_size(input_size)),
ControlInput::MouseRelative {dx,dy} => input_worker.mouse_relative(*dx,*dy),
ControlInput::MouseScroll {dx,dy} => input_worker.mouse_scroll(*dx,*dy),
ControlInput::Keyboard {key, pressed} => input_worker.windows_key_event(*key, *pressed),