Compare commits

..
10 Commits
Author SHA1 Message Date
VallierDevandClaude Opus 5.5 c38b41b543 docs: v0.2.2 notes include #40 (Mac keycodes, FFmpeg 4.4 decoding)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 18:59:18 +08:00
wanglz111 3d4d8d1d64 fix(viewer): use native keycodes for Mac targets; ffmpeg 4.4 compatible decoding (#40)
* fix(viewer): send native keycodes to Mac targets

* fix(viewer): reject unknown keyboard target platforms

* fix(viewer): support FFmpeg 4.4 passthrough decoding
2026-10-17 18:58:57 +08:00
VallierDevandClaude Opus 5.5 843e4aa342 Release v0.2.2
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 18:57:22 +08:00
VallierDevandClaude Opus 5.5 73c1d9b54d viewer: also send ClipboardTextChangeRequest for official Windows hosts
The official Windows host only acknowledges the rich format-list offer
and never asks for the data. Each local change now also sends RpcRequest
case 10 ClipboardTextChangeRequest with the text (its
handle_text_change_request -> notifyReceiveText path); the host answers
case 6 err=1 (CB_SUCCEEDED), logged as a status only. This project's
host ignores case 10 and keeps using the rich offer. Note: on the tested
Windows host the acknowledged text still did not reach the desktop
clipboard; under investigation.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 18:56:26 +08:00
VallierDevandClaude Opus 5.5 432593083e viewer: terminal on the official Windows host
Live against an official Windows host (PowerShell via ConPTY):
- Its rpc_main.proto has no check_env (requests end at 25), so a missing
  answer within 3 s no longer aborts; a negative answer still does.
- OPEN carries "option":"none"; the host rejects an OPEN without it
  ("terminal open: missing 'option' field", kError 1004). This
  project's host ignores the key.
- The host answers OPEN with {code,message}, header session 0, and keeps
  one terminal per connection as session 0; a nonzero header still
  overrides the body for this project's host.
- Echo requests whose SimpleAction omits the default action 0 are
  answered too.
Create/reattach on this project's host unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 18:36:45 +08:00
VallierDevandClaude Opus 5.5 e5e6a04b5d viewer: port mapping through the official Windows host
The official Windows host answers every PortMappingFrame with session_id
1 regardless of the viewer's value, so the random 64-bit session never
matched and every stream ended with a SYN timeout. One port-mapping peer
carries one session; use 1 (this project's host echoes it). Stream
failures and unmatched frames are now logged with reasons and wire types
only. Verified: HTTP through the official Windows host and this
project's host.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 17:52:05 +08:00
VallierDevandClaude Opus 5.5 23aec299bb viewer: upload to and download from the official Windows host
Live against an official Windows host:
- Uploads split Windows targets (C:\dir\name) on either separator and
  keep the host's separator for the directory; '/x' keeps the root.
- The upload request carries FileInfo.modified_time, the zlib FileList in
  list_data (field 6) and a task_unique_id, as official viewers with the
  list-compress feature do; the Windows host rejected the inline-only
  request with code 2. The inline copy stays for older hosts.
- Downloads accept blocks up to 256 KiB: the Windows host sent a 64 KiB
  file as one block, which the old 32 KiB limit rejected with code 3.
64 KiB and 1 MiB round trips matched by sha256 on the Windows host and
on this project's host.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 17:33:34 +08:00
VallierDevandClaude Opus 5.5 9a91000e0a docs: H.265 and 4:4:4 tested live against this project's host
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 17:17:19 +08:00
VallierDevandClaude Opus 5.5 693660ed85 viewer: decode H.265 and request 4:4:4
Offer H.265 (RFC 7798) next to H.264 and list H.265 and 4:4:4 decoder
caps; the host picks the codec. Depacketize single NAL, AP and FU units
with the existing reorder/loss handling and decode with the ffmpeg CLI
(-f hevc). UU_VIEWER_CHROMA=444 requests true color via
CaptureSettingRequest field 7 and follows CaptureSettingSync;
UU_VIEWER_CODEC=h264 keeps H.264 only.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 17:11:45 +08:00
VallierDevandClaude Opus 5.5 5c5958115a viewer: keep the open device page usable across list refreshes
The device list refresh (startup and every 30 s) reset the selected
device to none while its page stayed open, so every button on that page
silently did nothing. Re-select the same device by id after a refresh.
The replace-session confirmation now uses Chinese button labels like the
other dialogs instead of locale-dependent OK/Cancel.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-17 17:08:46 +08:00
28 changed files with 690 additions and 92 deletions

No files matched your search

Generated
+1 -1
View File
@@ -2741,7 +2741,7 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uu-remote-linux"
version = "0.2.1"
version = "0.2.2"
dependencies = [
"async-trait",
"base64 0.22.1",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "uu-remote-linux"
version = "0.2.1"
version = "0.2.2"
edition = "2021"
default-run = "uuremote-server"
authors = ["uu-remote-linux contributors"]
+17 -5
View File
@@ -425,19 +425,31 @@ terminal emulator (`TERMINAL` selects one). From source: `sudo apt install libsd
`assist CONNECT_ID` (the code is read from one stdin line, never an argument). See `uuremote-viewer --help`.
- Ctrl+Alt+F in the desktop window opens the files window on the same connection. `UU_VIEWER_AUDIO=0` disables sound,
`UU_VIEWER_CLIPBOARD=0` the clipboard, `UU_VIEWER_VIEW_ONLY=1` input; `UU_VIEWER_FFMPEG` selects the ffmpeg binary.
- Desktop offers H.264 / H.265; the host chooses the codec. `UU_VIEWER_CODEC=h264` limits both SDP and decoder capabilities to H.264.
The default request stays 4:2:0; `UU_VIEWER_CHROMA=444` requests true color (4:4:4) after CONTROL opens. The host may fall back to
4:2:0, as reported by CaptureSettingSync. FFmpeg converts either chroma format to RGB24 for display.
Mac targets use native macOS keycodes; Windows / Linux targets retain Windows VK. Keys follow physical positions, without
automatically translating Ctrl to Command; on a Mac target the PC Super / Windows key acts as Command. Only verified target
platform values 1 (Windows / this project's Linux host) and 4 (Mac) are accepted; unknown values fail explicitly. Account
connections look up the selected device only when the platform is absent; assistance requires it in the join response.
View-only sessions skip this lookup. Media keys and some JIS-specific keys are not mapped; Mac platform detection for remote
assistance and ISO / JIS keyboards have not been tested live.
**Tested live** (author's setup, not full acceptance):
| Feature | Official Windows host | Official Mac host | This project's Linux host |
|---|---|---|---|
| Desktop video (H.264) | ✅ | ✅ | ✅ |
| H.265, 4:4:4 true color | — | — | ✅ (VM, libx265) |
| Keyboard, mouse | ✅ | — | ✅ |
| File list / download / upload | ✅ list, download | — | ✅ |
| Port mapping, terminal | — | — | ✅ |
| Clipboard (text, both ways), sound | — | — | ✅ (VM) |
| File list / download / upload | ✅ | — | ✅ |
| Port mapping, terminal | ✅ (PowerShell terminal) | — | ✅ |
| Clipboard (text) | host → viewer only; viewer → Windows not applied yet | — | ✅ both ways (VM) |
| Sound | unconfirmed (no audio packets while the host is silent) | — | ✅ (VM) |
| Remote assistance (ID + code) | — | — | ✅ (VM, guest host) |
| Terminal session list (create, double-click reattach, end) | — | — | ✅ (VM) |
| Reboot, shutdown | — | — | ✅ (VM) |
| Wayland desktop window and clipboard (Hyprland) | — | — | ✅ (VM) |
| Wayland desktop window and clipboard | — | — | ✅ Hyprland, KDE; GNOME window works, clipboard unavailable (#37) |
"—" = not tested yet. Only use reboot/shutdown on machines you can restart. **Not tested live**: GNOME/KDE Wayland windows and clipboard, H.265 and 4:4:4 (only H.264 4:2:0 is decoded), multiple monitors. Logs carry field
"—" = not tested yet. Only use reboot/shutdown on machines you can restart. **Not tested live**: H.265 / 4:4:4 with official hosts, multiple monitors. Logs carry field
numbers, sizes and states only, never keystrokes, clipboard or file contents, codes or credentials.
+15 -6
View File
@@ -389,19 +389,28 @@ xfce4-session xfce4-panel xfdesktop xfwm4 gnome-terminal`。
`assist CONNECT_ID`(验证码从 stdin 读取一行,不能作为参数传入)。`uuremote-viewer --help` 查看全部参数。
- 桌面窗口中按 Ctrl+Alt+F 打开文件窗口(复用同一连接)。`UU_VIEWER_AUDIO=0` 关闭声音,`UU_VIEWER_CLIPBOARD=0` 关闭剪贴板,
`UU_VIEWER_VIEW_ONLY=1` 只看不控,`UU_VIEWER_FFMPEG` 指定 ffmpeg 路径。
- 桌面同时提供 H.264 / H.265,由主机协商选择;`UU_VIEWER_CODEC=h264` 将 SDP 和解码能力限制为 H.264。
默认请求 4:2:0;`UU_VIEWER_CHROMA=444` 在 CONTROL 打开后请求真彩(4:4:4)。主机可能回落到 4:2:0,
以 CaptureSettingSync 回报为准。两种色度均由 FFmpeg 转为 RGB24 显示。
Mac 目标使用 macOS 原生键码,Windows / Linux 目标保留 Windows VK。键盘按物理键位置转发,不自动把 Ctrl 改成 Command;
控制 Mac 时,PC 键盘的 Super / Win 键对应 Command。仅接受已验证的目标平台值 1(Windows / 本项目 Linux 主机)和 4(Mac),
未知值会报错,不猜测键码。账号连接缺少平台字段时查询目标设备;远程协助必须由连接响应提供平台。只看模式不需要此识别。
媒体键和部分 JIS 专用键尚未映射;远程协助的 Mac 平台识别及 ISO / JIS 键盘尚未实机验收。
**已实测**(作者环境,非全面验收):
| 功能 | 官方 Windows 被控端 | 官方 Mac 被控端 | 本项目 Linux 主机 |
|---|---|---|---|
| 桌面画面(H.264) | ✅ | ✅ | ✅ |
| H.265、4:4:4 真彩 | — | — | ✅(虚拟机,libx265) |
| 键盘、鼠标 | ✅ | — | ✅ |
| 文件列表 / 下载 / 上传 | ✅ 列表、下载 | — | ✅ |
| 端口映射、终端 | — | — | ✅ |
| 剪贴板(文本,双向)、声音 | — | — | ✅(虚拟机) |
| 文件列表 / 下载 / 上传 | ✅ | — | ✅ |
| 端口映射、终端 | ✅(终端为 PowerShell) | — | ✅ |
| 剪贴板(文本) | 仅被控端 → 控制端;控制端 → Windows 未生效 | — | ✅ 双向(虚拟机) |
| 声音 | 未确认(被控端未播放声音时没有音频包) | — | ✅(虚拟机) |
| 远程协助(ID + 验证码) | — | — | ✅(虚拟机,访客主机) |
| 终端会话列表(新建、双击接回、结束) | — | — | ✅(虚拟机) |
| 重启、关机 | — | — | ✅(虚拟机) |
| Wayland 下的桌面窗口与剪贴板(Hyprland) | — | — | ✅(虚拟机) |
| Wayland 下的桌面窗口与剪贴板 | — | — | ✅ Hyprland、KDE;GNOME 窗口正常,剪贴板不可用(#37) |
「—」表示尚未测试。重启/关机只应在可以重启的机器上使用。**未实测**:GNOME/KDE Wayland 下的窗口与剪贴板、
H.265 与 4:4:4(目前只解码 H.264 4:2:0)、多显示器。日志只记录字段号、大小和状态,不记录键盘、剪贴板、文件内容、验证码或凭据。
「—」表示尚未测试。重启/关机只应在可以重启的机器上使用。**未实测**:官方被控端的 H.265 / 4:4:4、多显示器。日志只记录字段号、大小和状态,不记录键盘、剪贴板、文件内容、验证码或凭据。
+6 -1
View File
@@ -658,6 +658,9 @@ int uu_viewer_ui_start(void) {
/// Lines of "id\tname\tplatform\tonline\tcurrent".
void uu_viewer_ui_set_devices(const char *tsv) {
pthread_mutex_lock(&d_lock);
/* A refresh (every 30 s) must keep the open device page usable: re-select by id, not reset. */
char keep[sizeof devices[0].id] = "";
if (selected >= 0 && selected < device_count) snprintf(keep, sizeof keep, "%s", devices[selected].id);
device_count = 0;
char *copy = strdup(tsv ? tsv : ""), *save = NULL;
for (char *line = strtok_r(copy, "\n", &save); line && device_count < 64; line = strtok_r(NULL, "\n", &save)) {
@@ -670,6 +673,7 @@ void uu_viewer_ui_set_devices(const char *tsv) {
}
free(copy);
selected = -1;
for (int i = 0; keep[0] && i < device_count; i++) if (!strcmp(devices[i].id, keep)) { selected = i; break; }
pthread_mutex_unlock(&d_lock);
if (init_state == 1) g_idle_add(apply_devices, NULL);
}
@@ -699,7 +703,8 @@ static int replace_result;
static gboolean confirm_replace(gpointer d) {
(void)d;
GtkWidget *dialog = gtk_message_dialog_new(GTK_WINDOW(window), GTK_DIALOG_MODAL | GTK_DIALOG_DESTROY_WITH_PARENT,
GTK_MESSAGE_QUESTION, GTK_BUTTONS_OK_CANCEL, "打开后将断开当前与该设备的连接,继续吗?");
GTK_MESSAGE_QUESTION, GTK_BUTTONS_NONE, "打开后将断开当前与该设备的连接,继续吗?");
gtk_dialog_add_buttons(GTK_DIALOG(dialog), "取消", GTK_RESPONSE_CANCEL, "继续", GTK_RESPONSE_OK, NULL);
int accepted = gtk_dialog_run(GTK_DIALOG(dialog)) == GTK_RESPONSE_OK;
gtk_widget_destroy(dialog);
pthread_mutex_lock(&init_lock); replace_result = accepted; pthread_cond_signal(&init_cond); pthread_mutex_unlock(&init_lock);
+30
View File
@@ -0,0 +1,30 @@
控制端:支持 H.265 / 4:4:4,可对官方 Windows 被控端上传、端口映射和终端。/ Viewer: H.265 / 4:4:4, and upload, port mapping and terminal with the official Windows host.
### 控制端新增 / Viewer — added
- **H.265 与 4:4:4 真彩**:桌面同时提供 H.264 / H.265,由被控端选择;`UU_VIEWER_CHROMA=444` 请求真彩,`UU_VIEWER_CODEC=h264` 只用 H.264。
已在虚拟机中对本项目主机实测(libx265 4:2:0 与 4:4:4、强制 H.264 均 0 解码错误);⚠️ 官方被控端的 H.265 / 4:4:4 未实测。
H.265 and 4:4:4 true color (tested against this project's host in a VM; not yet with official hosts).
### 控制端修复 / Viewer — fixed(均为实测发现并验证 / found and verified live)
- **官方 Windows 被控端上传文件**:之前上传会被拒绝(错误码 2)。现在按官方客户端的格式发送(Windows 路径、修改时间、压缩文件列表),
64 KiB 与 1 MiB 文件上传后再下载回来 sha256 一致。下载也不再拒绝大于 32 KiB 的数据块(错误码 3)。
- **官方 Windows 被控端端口映射**:之前每个连接都超时;现在可正常转发(经被控端访问 HTTP 返回 200)。
- **官方 Windows 被控端终端**:之前连上后没有任何输出;现在可打开 PowerShell 并交互。
- **启动窗口**:设备列表每 30 秒刷新后,已打开的设备页按钮全部失效的问题;「断开当前连接」确认框改为中文按钮。
- **连接官方 Mac 被控端时键盘错位**(#39,感谢 @wanglz111,#40):按 1 出空格、Backspace 出 c。Mac 目标改用 macOS 原生键码
(按物理位置转发,PC 的 Win 键对应 Command),Windows 与本项目主机仍用 Windows VK;无法确定目标平台时拒绝控制而不是乱发键码。
贡献者已在官方 Mac 被控端实机确认;⚠️ ISO/JIS 键盘、媒体键与 Mac 远程协助未验收。
- **FFmpeg 4.4 下没有画面**(#40):v0.2.0 / v0.2.1 的控制端解码参数 `-fps_mode` 不被 FFmpeg 4.4(如 Ubuntu 22.04 自带版本)识别,
解码进程直接退出。现改用新旧版本都支持的参数。
- 对本项目主机的上传、端口映射、终端新建/接回均已回归验证。
- Upload, port mapping and terminal now work with the official Windows host; launcher device page stays usable after list refreshes.
### 已知问题 / Known issues
- 控制端 → 官方 Windows 被控端的剪贴板:被控端确认收到文本,但未出现在其桌面剪贴板中,仍在排查(被控端 → 控制端正常)。
- GNOME Wayland 下控制端剪贴板不可用,且轮询会让 Dock 图标跳动(#37,修复中);窗口本身正常。KDE、Hyprland 正常。
- Viewer → Windows host clipboard is acknowledged but not applied yet; GNOME clipboard (#37).
### 下载 / Download
`uu-remote-linux-v0.2.2-x86_64-linux.tar.gz`:主机 `uuremote-server`(X11 版)与控制端 `uuremote-viewer`(x86_64,glibc ≥ 2.35;控制端需 SDL2、GTK3、ffmpeg),附带 `contrib/`。
**非官方项目,与网易无关。** / **Unofficial, not affiliated with NetEase.**
+99 -1
View File
@@ -14,9 +14,16 @@ pub struct JoinData {
#[serde(default)] pub report_server_address: String,
#[serde(default)] pub streamer_retry_delta_ms: i32,
#[serde(default)] pub international_connect: bool,
#[serde(default)] pub publisher_platform: Option<i32>,
}
fn device_platform(data:&Value,device:&str)->Option<i64>{
["my_binded_devices","others_shared_devices"].into_iter()
.filter_map(|key|data.get(key).and_then(Value::as_array)).flatten()
.find(|d|d.get("device_id").and_then(Value::as_str)==Some(device))
.and_then(|d|d.get("platform")).and_then(Value::as_i64)
}
#[derive(Debug, PartialEq, Eq)]
pub enum Error { AuthenticationExpired, Api(i32, String), InvalidResponse, Transport, MissingRoom, InvalidDevice, InvalidAssist }
pub enum Error { AuthenticationExpired, Api(i32, String), InvalidResponse, Transport, MissingRoom, InvalidDevice, InvalidAssist, UnknownKeyboardPlatform }
impl std::fmt::Display for Error {
// The server's short message is user-facing (e.g. 2002 「被控端正在被远控中,无法发起连接」).
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
@@ -27,6 +34,7 @@ impl std::fmt::Display for Error {
Error::InvalidAssist => write!(f, "请输入对方的设备 ID 和有效验证码"),
Error::InvalidResponse => write!(f, "服务器响应格式无效"),
Error::MissingRoom => write!(f, "服务器未返回可用的房间或信令信息"),
Error::UnknownKeyboardPlatform => write!(f, "无法确定目标键盘平台,不能启用控制"),
Error::Api(code, _) => write!(f, "连接请求失败(错误码 {code})"),
_ => write!(f, "viewer API: {self:?}"),
}
@@ -154,6 +162,21 @@ impl Api {
if status != 200 || body.get("code").and_then(Value::as_i64) != Some(0) { return Err(Error::InvalidResponse); }
body.get("data").cloned().ok_or(Error::InvalidResponse)
}
pub async fn target_is_mac(&self,room:&JoinData,device:Option<&str>)->Result<bool,Error>{
let platform=match room.publisher_platform {
Some(platform)=>Some(i64::from(platform)),
None=>match device {
Some(device)=>device_platform(&self.device_list().await?,device),
None=>None,
},
};
// Observed device/share platforms: 1 = Windows (also our Linux host), 4 = Mac.
match platform {
Some(1)=>Ok(false),
Some(4)=>Ok(true),
_=>Err(Error::UnknownKeyboardPlatform),
}
}
/// Method and wire-key mapping remain unverified. Caller must select both explicitly.
pub async fn devices(&self, method: reqwest::Method, array_pointer: &str) -> Result<Vec<Value>, Error> {
let (status, body) = self.request(method, "/api/v1/device/list", None).await?;
@@ -167,6 +190,16 @@ impl Api {
mod tests {
use super::*;
#[test]
fn target_platform_matches_only_the_selected_device(){
let data=json!({"my_binded_devices":[{"device_id":"windows","platform":1},{"device_id":"mac","platform":4}],
"others_shared_devices":[{"device_id":"shared-mac","platform":4}]});
assert_eq!(device_platform(&data,"windows"),Some(1));
assert_eq!(device_platform(&data,"mac"),Some(4));
assert_eq!(device_platform(&data,"shared-mac"),Some(4));
assert_eq!(device_platform(&data,"missing"),None);
assert_eq!(device_platform(&json!({}),"mac"),None);
}
#[test]
fn assistance_response_validation_and_safe_user_errors() {
let mut fixture:Value=serde_json::from_str(include_str!("../../tests/fixtures/viewer/join.json")).unwrap();
fixture["data"]["publisher_device_id"]=json!("synthetic-publisher");
@@ -188,6 +221,11 @@ mod tests {
let fixture: Value = serde_json::from_str(include_str!("../../tests/fixtures/viewer/join.json")).unwrap();
let data = parse_join(200, fixture.clone()).unwrap();
assert_eq!(data.signaling_list.len(), 1);
assert_eq!(data.publisher_platform,None);
let mut empty=fixture.clone();empty["data"]["publisher_platform"]=Value::Null;
assert_eq!(parse_join(200,empty).unwrap().publisher_platform,None);
let mut mac=fixture.clone();mac["data"]["publisher_platform"]=json!(4);
assert_eq!(parse_join(200,mac).unwrap().publisher_platform,Some(4));
for key in ["report_token", "report_url"] {
let mut v = fixture.clone(); v["data"].as_object_mut().unwrap().remove(key);
assert!(matches!(parse_join(200,v), Err(Error::InvalidResponse)));
@@ -242,6 +280,62 @@ mod http_tests {
(url,job)
}
fn signer()->UUAuthClient {UUAuthClient::new(Credentials{token:"synthetic-auth".into(),sign_key:"synthetic-key".into(),..Default::default()})}
fn room(platform:Option<i32>)->JoinData {
let mut room=parse_join(200,serde_json::from_str(include_str!("../../tests/fixtures/viewer/join.json")).unwrap()).unwrap();
room.publisher_platform=platform;room
}
#[tokio::test]
async fn keyboard_platform_uses_room_metadata_without_device_lookup(){
let api=Api::new(signer(),"http://127.0.0.1:1");
for device in [None,Some("synthetic-device")] {
assert_eq!(api.target_is_mac(&room(Some(1)),device).await,Ok(false));
assert_eq!(api.target_is_mac(&room(Some(4)),device).await,Ok(true));
for platform in [0,-1,2,3,5,999,i32::MIN,i32::MAX] {
assert_eq!(api.target_is_mac(&room(Some(platform)),device).await,Err(Error::UnknownKeyboardPlatform));
}
}
assert_eq!(api.target_is_mac(&room(None),None).await,Err(Error::UnknownKeyboardPlatform));
}
#[tokio::test]
async fn keyboard_platform_looks_up_only_the_selected_account_device(){
for list in ["my_binded_devices","others_shared_devices"] {
for (platform,mac) in [(1,false),(4,true)] {
let data=json!({list:[{"device_id":"unrelated","platform":4},
{"device_id":"synthetic-device","platform":platform}]});
let(url,job)=server(vec![(200,json!({"code":0,"data":data}))]).await;
assert_eq!(Api::new(signer(),&url).target_is_mac(&room(None),Some("synthetic-device")).await,Ok(mac));
let requests=job.await.unwrap();assert_eq!(requests.len(),1);
assert!(requests[0].starts_with("GET /api/v1/device/list "));
}
}
}
#[tokio::test]
async fn keyboard_platform_rejects_unknown_or_malformed_device_metadata(){
let mut devices=vec![json!({"device_id":"unrelated","platform":4}),json!({"device_id":"synthetic-device"})];
for platform in [json!(0),json!(-1),json!(999),json!("4"),json!(true),json!(4.5),json!(null),json!(u64::MAX)] {
devices.push(json!({"device_id":"synthetic-device","platform":platform}));
}
for device in devices {
let data=json!({"my_binded_devices":[device]});
let(url,job)=server(vec![(200,json!({"code":0,"data":data}))]).await;
assert_eq!(Api::new(signer(),&url).target_is_mac(&room(None),Some("synthetic-device")).await,Err(Error::UnknownKeyboardPlatform));
assert_eq!(job.await.unwrap().len(),1);
}
}
#[tokio::test]
async fn keyboard_platform_propagates_device_lookup_failures(){
for (status,body,error) in [
(422,json!({"code":1120}),Error::AuthenticationExpired),
(503,json!({"code":5000}),Error::InvalidResponse),
(200,json!({"code":0}),Error::InvalidResponse),
] {
let(url,job)=server(vec![(status,body)]).await;
assert_eq!(Api::new(signer(),&url).target_is_mac(&room(None),Some("synthetic-device")).await,Err(error));
assert_eq!(job.await.unwrap().len(),1);
}
let api=Api::new(signer(),"http://127.0.0.1:1");
assert_eq!(api.target_is_mac(&room(None),Some("synthetic-device")).await,Err(Error::Transport));
}
#[tokio::test]
async fn signed_join_and_explicit_device_list_use_only_loopback_http() {
let fixture=serde_json::from_str(include_str!("../../tests/fixtures/viewer/join.json")).unwrap();
@@ -325,6 +419,10 @@ mod strict_tests {
for (key,bad) in [("token",json!(false)),("signaling_list",json!([7])),("ws_connect_timeout_ms",json!(2147483648u64)),("streamer_retry_delta_ms",json!(null)),("international_connect",json!(1)),("report_token",json!(null)),("report_url",json!(7)),("report_server_address",json!([]))] {
let mut v=fixture.clone();v["data"][key]=bad;assert!(matches!(parse_join(200,v),Err(Error::InvalidResponse)));
}
for platform in [json!("4"),json!(true),json!(4.5),json!(2147483648u64),json!(-2147483649i64)] {
let mut v=fixture.clone();v["data"]["publisher_platform"]=platform;
assert!(matches!(parse_join(200,v),Err(Error::InvalidResponse)));
}
for key in ["code","msg"] {
let mut v=fixture.clone();v.as_object_mut().unwrap().remove(key);assert!(matches!(parse_join(200,v),Err(Error::InvalidResponse)));
}
+24 -5
View File
@@ -63,10 +63,16 @@ impl SyncState {
self.last=Some(value);Ok(())
}
fn next(&mut self)->u64{self.seq=self.seq.wrapping_add(1).max(100);self.seq}
fn local(&mut self,value:String)->Result<Option<Vec<u8>>,&'static str>{
/// Two requests per change: the rich format-list offer (this project's host asks for the data) and
/// RpcRequest case 10 ClipboardTextChangeRequest carrying the text, which the official Windows host applies
/// (handle_text_change_request -> notifyReceiveText); it only acknowledges the rich offer. Each host
/// ignores the other form.
fn local(&mut self,value:String)->Result<Option<Vec<Vec<u8>>>,&'static str>{
if self.last.as_ref()==Some(&value){return Ok(None)}
let id=self.next();let payload=self.rich.offer(id,value.clone())?;
self.last=Some(value);Ok(Some(wire::envelope(self.next(),21,&payload)))
let rich=wire::envelope(self.next(),21,&payload);
let text=wire::envelope(self.next(),21,&crate::clipboard_protocol::text_request(self.next(),&value));
self.last=Some(value);Ok(Some(vec![rich,text]))
}
fn receive(&mut self,data:&[u8])->Result<(Vec<Vec<u8>>,Option<String>),&'static str>{
if data.len()>MAX_TEXT*2+1024{return Err("clipboard message exceeds limit")}
@@ -85,7 +91,14 @@ impl SyncState {
value=Some(text);replies.push(wire::envelope(self.next(),22,&text::text_response(&header)));
}
},
22=>if let Some(text)=self.rich.response(payload)?{value=Some(text)},
22=>{
// TextChangeResponse (case 6 {1 err}, CB_SUCCEEDED=1) answers our case 10 request: status only.
if let Some(status)=decode_wire(payload).ok().and_then(|f|f.into_iter().find_map(|x|match (x.number,x.value){(6,WireValue::Bytes(b))=>Some(b.to_vec()),_=>None})){
let err=decode_wire(&status).ok().and_then(|f|f.into_iter().find_map(|x|match (x.number,x.value){(1,WireValue::Varint(v))=>Some(v),_=>None})).unwrap_or(0);
log::info!("Viewer clipboard text change acknowledged: err={err}");
}
if let Some(text)=self.rich.response(payload)?{value=Some(text)}
},
_=>{},
}
}
@@ -143,7 +156,7 @@ impl Bridge {
if channel.ready_state()!=webrtc::data_channel::data_channel_state::RTCDataChannelState::Open{continue}
match backend.read().await {
Ok(Some(value))=>{let length=value.len();match state.local(value){
Ok(Some(message))=>{if send(&channel,message).await.is_err(){break}log::info!("Viewer local clipboard offered: bytes={length}");},
Ok(Some(messages))=>{let mut failed=false;for m in messages{if send(&channel,m).await.is_err(){failed=true;break}}if failed{break}log::info!("Viewer local clipboard offered: bytes={length}");},
Ok(None)=>{},Err(_)=>{},
}},
Ok(None)=>{},Err(())=>{warn();break},
@@ -203,7 +216,7 @@ impl Drop for Bridge{fn drop(&mut self){self.stop();}}
fn native_host_exchange(outbound:usize,reverse:bool)->(Option<String>,bool){
let mut viewer=SyncState::default();let mut host=[rich::State::default(),rich::State::default()];let mut id=1000;
let value="clipboard-中文🙂\n".to_owned();
let offer=if reverse{wire::envelope(1,21,&host[1].offer(1,value.clone()).unwrap())}else{viewer.local(value.clone()).unwrap().unwrap()};
let offer=if reverse{wire::envelope(1,21,&host[1].offer(1,value.clone()).unwrap())}else{viewer.local(value.clone()).unwrap().unwrap().remove(0)};
let mut queue=std::collections::VecDeque::from([(reverse,if reverse{1}else{outbound},offer)]);
let mut received=None;let mut unavailable=false;let mut steps=0;
while let Some((to_viewer,channel,packet))=queue.pop_front(){
@@ -231,6 +244,12 @@ impl Drop for Bridge{fn drop(&mut self){self.stop();}}
if reverse && received.is_some(){viewer.last=Some(value.clone());assert!(viewer.local(value).unwrap().is_none());}
(received,unavailable)
}
#[test] fn local_change_also_sends_windows_text_change_request(){
let mut viewer=SyncState::default();
let messages=viewer.local("windows 中文".into()).unwrap().unwrap();assert_eq!(messages.len(),2);
let rpc=decode_wire(&messages[1]).unwrap().into_iter().find_map(|f|match (f.number,f.value){(21,WireValue::Bytes(b))=>Some(b.to_vec()),_=>None}).unwrap();
let (_,text)=crate::clipboard_protocol::read_text_request(&rpc).unwrap().unwrap();assert_eq!(text,"windows 中文");
}
#[test] fn control_clipboard_reproduces_native_host_failures(){
assert_eq!(native_host_exchange(0,false),(None,false)); // Complete transfer, no CONTROL desktop sink.
assert_eq!(native_host_exchange(0,true),(None,true)); // TEXT offer, empty CONTROL provider.
+14 -7
View File
@@ -1,29 +1,36 @@
//! Uses the same optional FFmpeg dependency as the host inproc-av feature.
use super::desktop::{Decoder,Frame};
use ffmpeg_next as av;
pub struct Ffmpeg { decoder:av::decoder::Video }
use super::media::Codec;
pub struct Ffmpeg { decoder:av::decoder::Video, codec:Codec }
impl Ffmpeg {
pub fn new()->Result<Self,String> {
Self::with_codec(Codec::H264)
}
fn with_codec(kind:Codec)->Result<Self,String>{
av::init().map_err(|_|"FFmpeg initialization failed")?;
// Library diagnostics can contain bitstream data; suppress them globally in this viewer process.
av::log::set_level(av::log::Level::Quiet);
let codec=av::decoder::find(av::codec::Id::H264).ok_or("H264 decoder unavailable")?;
let decoder=av::codec::context::Context::new().decoder().open_as(codec).and_then(|d|d.video()).map_err(|_|"H264 decoder initialization failed")?;
Ok(Self{decoder})
let codec=av::decoder::find(match kind {Codec::H264=>av::codec::Id::H264,Codec::H265=>av::codec::Id::HEVC}).ok_or("Video decoder unavailable")?;
let decoder=av::codec::context::Context::new().decoder().open_as(codec).and_then(|d|d.video()).map_err(|_|"Video decoder initialization failed")?;
Ok(Self{decoder,codec:kind})
}
}
impl Decoder for Ffmpeg {
fn set_codec(&mut self,codec:Codec)->Result<(),String>{
if self.codec!=codec {*self=Self::with_codec(codec)?;}Ok(())
}
fn reset(&mut self){self.decoder.flush();}
fn decode(&mut self,au:&[u8])->Result<Vec<Frame>,String>{
self.decoder.send_packet(&av::Packet::copy(au)).map_err(|_|"H264 packet rejected")?;
self.decoder.send_packet(&av::Packet::copy(au)).map_err(|_|"Video packet rejected")?;
let mut frames=Vec::new();let mut decoded=av::frame::Video::empty();
loop {
match self.decoder.receive_frame(&mut decoded) {
Ok(())=>{},
Err(av::Error::Other{errno}) if errno==libc::EAGAIN=>break,
Err(_)=>return Err("H264 decode failed".into()),
Err(_)=>return Err("Video decode failed".into()),
}
if decoded.is_corrupt(){return Err("Corrupt H264 frame".into())}
if decoded.is_corrupt(){return Err("Corrupt video frame".into())}
let (w,h)=(decoded.width(),decoded.height());
if w==0 || h==0 || w>8192 || h>8192 {return Err("Decoded dimensions exceed limit".into())}
let mut scaler=av::software::scaling::context::Context::get(decoded.format(),w,h,av::format::Pixel::RGB24,w,h,av::software::scaling::flag::Flags::BILINEAR).map_err(|_|"Frame conversion unavailable")?;
+51 -8
View File
@@ -1,12 +1,13 @@
//! H.264 decoding through an external `ffmpeg` process, like the host's encoder pipeline.
//! Needs only the ffmpeg CLI (any recent version), not FFmpeg development libraries.
//! H.264/H.265 decoding through an external `ffmpeg` process, like the host's encoder pipeline.
//! Needs only the ffmpeg CLI, not FFmpeg development libraries.
//! Output is scaled into a fixed letterboxed size so every frame has a known byte length.
use super::desktop::{Decoder,Frame};
use super::media::Codec;
use std::io::{Read,Write};
use std::process::{Child,ChildStdin,Command,Stdio};
use std::sync::mpsc::{Receiver,TryRecvError};
pub struct Cli { child:Child, input:ChildStdin, frames:Receiver<Vec<u8>>, width:u32, height:u32 }
pub struct Cli { child:Child, input:Option<ChildStdin>, codec:Codec, frames:Receiver<Vec<u8>>, width:u32, height:u32 }
impl Cli {
/// `UU_VIEWER_SIZE=WxH` picks the decoded size (default 1920x1080); `UU_VIEWER_FFMPEG` the binary.
@@ -15,15 +16,19 @@ impl Cli {
.and_then(|v|{let (w,h)=v.split_once('x')?;Some((w.parse().ok()?,h.parse().ok()?))})
.filter(|&(w,h):&(u32,u32)|(16..=8192).contains(&w)&&(16..=8192).contains(&h)&&w%2==0&&h%2==0)
.unwrap_or((1920,1080));
Self::with_codec(Codec::H264,width,height)
}
fn with_codec(codec:Codec,width:u32,height:u32)->Result<Self,String> {
let ffmpeg=std::env::var("UU_VIEWER_FFMPEG").unwrap_or_else(|_|"ffmpeg".into());
let filter=format!("scale={width}:{height}:force_original_aspect_ratio=decrease,pad={width}:{height}:(ow-iw)/2:(oh-ih)/2");
let mut child=Command::new(ffmpeg)
.args(["-hide_banner","-loglevel","error","-nostdin","-threads","1","-probesize","32","-analyzeduration","0",
"-fflags","nobuffer","-flags","low_delay","-f","h264","-i","pipe:0","-an","-vf",&filter,
"-fflags","nobuffer","-flags","low_delay","-f",codec.demuxer(),"-i","pipe:0","-an","-vf",&filter,
"-pix_fmt","rgb24",
// Raw H.264 carries no timestamps; constant-rate output dropped almost every frame of
// Raw H.264/H.265 carries no timestamps; constant-rate output dropped almost every frame of
// the official Mac host's stream (270 → 2). Pass frames through unchanged.
"-fps_mode","passthrough","-f","rawvideo","pipe:1"])
// -vsync 0 also supports Ubuntu 22.04's FFmpeg 4.4, which lacks -fps_mode.
"-vsync","0","-f","rawvideo","pipe:1"])
.stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::null())
.spawn().map_err(|e|format!("Start ffmpeg decoder: {e}"))?;
let input=child.stdin.take().ok_or("ffmpeg stdin missing")?;
@@ -38,14 +43,18 @@ impl Cli {
match tx.try_send(frame) { Ok(())|Err(std::sync::mpsc::TrySendError::Full(_))=>{}, Err(_)=>break }
}
}).map_err(|e|format!("Decoder reader: {e}"))?;
Ok(Self{child,input,frames,width,height})
Ok(Self{child,input:Some(input),codec,frames,width,height})
}
}
impl Decoder for Cli {
fn set_codec(&mut self,codec:Codec)->Result<(),String>{
if self.codec!=codec {*self=Self::with_codec(codec,self.width,self.height)?;}Ok(())
}
fn reset(&mut self) {}
fn decode(&mut self,au:&[u8])->Result<Vec<Frame>,String> {
self.input.write_all(au).and_then(|_|self.input.flush()).map_err(|_|"ffmpeg decoder exited")?;
let input=self.input.as_mut().ok_or("ffmpeg input closed")?;
input.write_all(au).and_then(|_|input.flush()).map_err(|_|"ffmpeg decoder exited")?;
let mut out=Vec::new();
loop {
match self.frames.try_recv() {
@@ -59,3 +68,37 @@ impl Decoder for Cli {
}
impl Drop for Cli { fn drop(&mut self) { let _=self.child.kill(); let _=self.child.wait(); } }
#[cfg(test)] mod tests {
use super::*;
// Six synthetic red IDRs per fixture; no captured desktop or external encoder dependency.
#[test] fn ffmpeg_decodes_h264_and_hevc_420_and_444_to_rgb24(){
match Command::new("ffmpeg").arg("-version").stdout(Stdio::null()).stderr(Stdio::null()).status(){
Err(e) if e.kind()==std::io::ErrorKind::NotFound=>{eprintln!("skip: ffmpeg is not on PATH");return},
result=>assert!(result.expect("ffmpeg availability").success()),
}
let mut cli=Cli::with_codec(Codec::H265,32,32).unwrap();
for (codec,data) in [
(Codec::H264,&include_bytes!("../../tests/fixtures/viewer/red-420.h264")[..]),
(Codec::H265,&include_bytes!("../../tests/fixtures/viewer/red-420.hevc")[..]),
(Codec::H264,&include_bytes!("../../tests/fixtures/viewer/red-444.h264")[..]),
(Codec::H265,&include_bytes!("../../tests/fixtures/viewer/red-444.hevc")[..]),
] {
cli.set_codec(codec).unwrap();
// Exercise the same streaming process and RGB conversion as SDL, then drain at EOF.
let mut frames=cli.decode(data).unwrap();drop(cli.input.take());
loop {match cli.frames.recv_timeout(std::time::Duration::from_secs(5)) {
Ok(rgb)=>frames.push(Frame{width:32,height:32,rgb}),
Err(std::sync::mpsc::RecvTimeoutError::Disconnected)=>break,
Err(e)=>panic!("{codec:?} decoder did not finish: {e}"),
}}
let status=cli.child.wait().unwrap();
assert!(status.success(),"{codec:?}: ffmpeg exited with {status}");
assert!(!frames.is_empty(),"{codec:?}: no RGB frames");
for frame in frames {
assert_eq!(frame.rgb.len(),32*32*3);
assert!(frame.rgb.chunks_exact(3).all(|p|p[0]>220 && p[1]<30 && p[2]<30),"{codec:?}: RGB conversion");
}
}
}
}
+6 -6
View File
@@ -1,19 +1,19 @@
use super::media::Latest;
use std::sync::{Arc, atomic::{AtomicBool,AtomicU32,Ordering}};
#[derive(Clone)]
pub struct Desktop { pub units:Latest<Vec<u8>>, pub pli:Arc<AtomicBool>, pub reset:Arc<AtomicBool>, pub ssrc:Arc<AtomicU32> }
pub struct Desktop { pub units:Latest<super::media::AccessUnit>, pub pli:Arc<AtomicBool>, pub reset:Arc<AtomicBool>, pub ssrc:Arc<AtomicU32> }
impl Default for Desktop {
fn default()->Self {Self{units:Latest::new(3),pli:Arc::new(AtomicBool::new(false)),reset:Arc::new(AtomicBool::new(false)),ssrc:Arc::new(AtomicU32::new(0))}}
}
pub struct Frame { pub width:u32, pub height:u32, pub rgb:Vec<u8> }
pub trait Decoder { fn decode(&mut self, au:&[u8])->Result<Vec<Frame>,String>; fn reset(&mut self); }
pub trait Decoder { fn set_codec(&mut self,codec:super::media::Codec)->Result<(),String>; fn decode(&mut self, au:&[u8])->Result<Vec<Frame>,String>; fn reset(&mut self); }
pub trait FrameSink { fn present(&mut self,frame:&Frame)->Result<(),String>; }
/// Counts only (access units, presented frames, decode errors) for the periodic stats line.
pub static STATS:[std::sync::atomic::AtomicU64;3]=[std::sync::atomic::AtomicU64::new(0),std::sync::atomic::AtomicU64::new(0),std::sync::atomic::AtomicU64::new(0)];
pub fn display(decoder:&mut dyn Decoder,sink:&mut dyn FrameSink,au:&[u8],pli:&AtomicBool)->Result<(),String> {
pub fn display(decoder:&mut dyn Decoder,sink:&mut dyn FrameSink,au:&[u8],codec:super::media::Codec,pli:&AtomicBool)->Result<(),String> {
STATS[0].fetch_add(1,Ordering::Relaxed);
// Local diagnostic: UU_VIEWER_DUMP_H264=path appends the first 300 access units (owner's own stream).
if STATS[0].load(Ordering::Relaxed)<=300 { if let Some(path)=std::env::var_os("UU_VIEWER_DUMP_H264") {
if codec==super::media::Codec::H264 && STATS[0].load(Ordering::Relaxed)<=300 { if let Some(path)=std::env::var_os("UU_VIEWER_DUMP_H264") {
use std::io::Write; if let Ok(mut f)=std::fs::OpenOptions::new().create(true).append(true).open(path){let _=f.write_all(au);}
}}
match decoder.decode(au) {
@@ -29,7 +29,7 @@ pub fn display(decoder:&mut dyn Decoder,sink:&mut dyn FrameSink,au:&[u8],pli:&At
}
#[cfg(test)] mod tests {
use super::*;
struct Mock(bool);impl Decoder for Mock {fn decode(&mut self,_:&[u8])->Result<Vec<Frame>,String>{if self.0{Err("synthetic".into())}else{Ok(vec![Frame{width:1,height:1,rgb:vec![0;3]}])}} fn reset(&mut self){self.0=false;}}
struct Mock(bool);impl Decoder for Mock {fn set_codec(&mut self,_:crate::viewer::media::Codec)->Result<(),String>{Ok(())} fn decode(&mut self,_:&[u8])->Result<Vec<Frame>,String>{if self.0{Err("synthetic".into())}else{Ok(vec![Frame{width:1,height:1,rgb:vec![0;3]}])}} fn reset(&mut self){self.0=false;}}
struct Sink(usize);impl FrameSink for Sink{fn present(&mut self,f:&Frame)->Result<(),String>{assert_eq!(f.rgb.len(),3);self.0+=1;Ok(())}}
#[test] fn decode_error_requests_pli_and_recovers(){let mut d=Mock(true);let mut s=Sink(0);let p=AtomicBool::new(false);display(&mut d,&mut s,&[],&p).unwrap();assert!(p.load(Ordering::Acquire));display(&mut d,&mut s,&[],&p).unwrap();assert_eq!(s.0,1);}
#[test] fn decode_error_requests_pli_and_recovers(){let mut d=Mock(true);let mut s=Sink(0);let p=AtomicBool::new(false);display(&mut d,&mut s,&[],crate::viewer::media::Codec::H264,&p).unwrap();assert!(p.load(Ordering::Acquire));display(&mut d,&mut s,&[],crate::viewer::media::Codec::H264,&p).unwrap();assert_eq!(s.0,1);}
}
+49 -2
View File
@@ -16,6 +16,25 @@ pub fn windows_vk(scan: i32) -> Option<i32> {
_=>return None,
})
}
pub fn mac_keycode(scan:i32)->Option<i32>{
// Inverted SDL Darwin table (release-2.32.10/src/events/scancodes_darwin.h).
// Live Mac host evidence: Windows VK 49 becomes Space, and VK 8 becomes C.
Some(match scan{
4=>0,22=>1,7=>2,9=>3,11=>4,10=>5,29=>6,27=>7,6=>8,25=>9,100=>10,5=>11,
20=>12,26=>13,8=>14,21=>15,28=>16,23=>17,30=>18,31=>19,32=>20,33=>21,
35=>22,34=>23,46=>24,38=>25,36=>26,45=>27,37=>28,39=>29,48=>30,18=>31,
24=>32,47=>33,12=>34,19=>35,40=>36,15=>37,13=>38,52=>39,14=>40,51=>41,
49=>42,54=>43,56=>44,17=>45,16=>46,55=>47,43=>48,44=>49,53=>50,42=>51,
41=>53,231=>54,227=>55,225=>56,57=>57,226=>58,224=>59,229=>60,230=>61,228=>62,
108=>64,99=>65,85=>67,87=>69,83=>71,84=>75,88=>76,86=>78,109=>79,110=>80,
103=>81,98=>82,89=>83,90=>84,91=>85,92=>86,93=>87,94=>88,95=>89,96=>91,97=>92,
62=>96,63=>97,64=>98,60=>99,65=>100,66=>101,68=>103,104=>105,107=>106,105=>107,
70=>105,71=>107,72=>113,
67=>109,101=>110,69=>111,106=>113,73=>114,74=>115,75=>116,76=>117,61=>118,
77=>119,59=>120,78=>121,58=>122,80=>123,79=>124,81=>125,82=>126,
_=>return None,
})
}
#[derive(Clone,Copy,Debug,PartialEq)]
pub struct Viewport { pub x:f64, pub y:f64, pub width:f64, pub height:f64 }
impl Viewport {
@@ -30,16 +49,17 @@ impl Viewport {
}
}
#[derive(Default)]
pub struct Input { view_only:bool, keys:BTreeMap<i32,i32>, buttons:BTreeSet<i32> }
pub struct Input { view_only:bool, mac:bool, keys:BTreeMap<i32,i32>, buttons:BTreeSet<i32> }
pub fn view_only(argument:Option<&str>,environment:Option<&str>)->Result<bool,String> {
match argument {None=>Ok(environment==Some("1")),Some("--view-only")=>Ok(true),_=>Err("Expected --view-only".into())}
}
impl Input {
pub fn new(view_only:bool)->Self {Self{view_only,..Default::default()}}
pub fn set_mac(&mut self,mac:bool){self.mac=mac;}
pub fn enabled(&self)->bool {!self.view_only}
pub fn key(&mut self,scan:i32,down:bool)->Option<Value> {
if self.view_only {return None}
let vk=windows_vk(scan)?;
let vk=if self.mac{mac_keycode(scan)}else{windows_vk(scan)}?;
let changed=if down {
if self.keys.contains_key(&scan) {return None}
let already=self.keys.values().any(|v|*v==vk);self.keys.insert(scan,vk);!already
@@ -65,6 +85,33 @@ impl Input {
}
#[cfg(test)] mod tests {
use super::*;
#[test] fn mac_target_uses_native_codes_and_releases_same_keys(){
let mut input=Input::default();input.set_mac(true);
for (scan,code) in [(30,18),(39,29),(42,51),(44,49),(6,8),(4,0),(40,36),(43,48),
(80,123),(79,124),(81,125),(82,126),(224,59),(225,56),(226,58),(227,55)]{
assert_eq!(input.key(scan,true).unwrap()["key"],code);
assert_eq!(input.key(scan,false).unwrap()["key"],code);
}
input.set_mac(false);assert_eq!(input.key(30,true).unwrap()["key"],49);
}
#[test] fn mac_focus_release_keeps_zero_keycode_and_modifiers(){
let mut input=Input::default();input.set_mac(true);
assert!(input.key(4,true).is_some());assert!(input.key(4,true).is_none());
input.key(225,true);input.key(227,true);
let release=input.release_all();
assert_eq!(release,vec![json!({"action":"kbd_release","key":0}),
json!({"action":"kbd_release","key":55}),json!({"action":"kbd_release","key":56})]);
assert!(input.release_all().is_empty());assert!(input.key(4,false).is_none());
let mut watch=Input::new(true);watch.set_mac(true);assert!(watch.key(4,true).is_none());
}
#[test] fn mac_function_key_aliases_release_after_last_keyup(){
for (pc,function,code) in [(70,104,105),(71,105,107),(72,106,113)]{
let mut input=Input::default();input.set_mac(true);
assert_eq!(input.key(pc,true).unwrap()["key"],code);
assert!(input.key(function,true).is_none());assert!(input.key(pc,false).is_none());
assert_eq!(input.key(function,false).unwrap()["key"],code);
}
}
#[test] fn watch_switch_is_explicit_and_validated(){
assert!(!view_only(None,None).unwrap());assert!(!view_only(None,Some("0")).unwrap());
assert!(view_only(None,Some("1")).unwrap());assert!(view_only(Some("--view-only"),Some("0")).unwrap());
+95 -5
View File
@@ -1,19 +1,32 @@
//! Bounded RFC 6184 packetization-mode 1 receiver. No media contents in logs.
//! Bounded RFC 6184 / RFC 7798 non-interleaved receivers (no DONL). No media contents in logs.
use std::{collections::VecDeque, sync::{Arc, Mutex}};
use std::time::{Duration,Instant};
#[derive(Clone,Copy,Debug,Default,PartialEq,Eq)]
pub enum Codec { #[default] H264, H265 }
impl Codec {
pub fn from_mime(mime:&str)->Option<Self> {
if mime.eq_ignore_ascii_case("video/H264"){Some(Self::H264)}
else if mime.eq_ignore_ascii_case("video/H265"){Some(Self::H265)}else{None}
}
pub fn demuxer(self)->&'static str {match self {Self::H264=>"h264",Self::H265=>"hevc"}}
}
pub struct AccessUnit {pub codec:Codec,pub data:Vec<u8>}
const MAX_AU: usize = 8 * 1024 * 1024;
#[derive(Clone)]
pub struct Packet { pub seq: u16, pub timestamp: u32, pub marker: bool, pub payload: Vec<u8> }
#[derive(Default)]
pub struct Output { pub units: Vec<Vec<u8>>, pub pli: bool }
pub struct Receiver {
codec: Codec,
next: Option<u16>, pending: Vec<Packet>, timestamp: Option<u32>,
au: Vec<u8>, fragment: Option<u8>, damaged: bool, gap_since:Option<Instant>, last_packet:Instant,
au: Vec<u8>, fragment: Option<u16>, damaged: bool, gap_since:Option<Instant>, last_packet:Instant,
}
impl Default for Receiver {
fn default() -> Self { Self { next: None, pending: Vec::new(), timestamp: None, au: Vec::new(), fragment: None, damaged: false, gap_since:None,last_packet:Instant::now() } }
fn default() -> Self { Self { codec: Codec::H264, next: None, pending: Vec::new(), timestamp: None, au: Vec::new(), fragment: None, damaged: false, gap_since:None,last_packet:Instant::now() } }
}
impl Receiver {
pub fn new(codec:Codec)->Self {Self{codec,..Self::default()}}
pub fn codec(&self)->Codec {self.codec}
pub fn push(&mut self, p: Packet) -> Output {
if p.payload.len()>65536 {return Output{units:Vec::new(),pli:true}}
let next = *self.next.get_or_insert(p.seq);
@@ -67,6 +80,7 @@ impl Receiver {
self.au.extend_from_slice(bytes); Ok(())
}
fn nal(&mut self, p: &[u8]) -> Result<(), ()> {
if self.codec==Codec::H265 {return self.hevc(p)}
let h = *p.first().ok_or(())?;
if h & 0x80 != 0 { return Err(()) }
match h & 31 {
@@ -86,14 +100,47 @@ impl Receiver {
let header = (h & 0xe0) | kind;
if start {
if self.fragment.is_some() { return Err(()) }
self.append(&[0,0,0,1,header])?; self.fragment = Some(header);
} else if self.fragment != Some(header) { return Err(()) }
self.append(&[0,0,0,1,header])?; self.fragment = Some(u16::from(header));
} else if self.fragment != Some(u16::from(header)) { return Err(()) }
self.append(&p[2..])?;
if end { self.fragment = None; } Ok(())
}
_ => Err(()),
}
}
fn hevc_header(p:&[u8])->Result<u16,()> {
if p.len()<2 || p[0]&0x80!=0 || p[1]&7==0 {return Err(())}
Ok(u16::from_be_bytes([p[0],p[1]]))
}
fn hevc(&mut self,p:&[u8])->Result<(),()> {
Self::hevc_header(p)?;
match (p[0]>>1)&63 {
0..=47 if self.fragment.is_none()=>{self.append(&[0,0,0,1])?;self.append(p)},
48 if self.fragment.is_none()=>{
let mut rest=&p[2..];if rest.is_empty(){return Err(())}
while !rest.is_empty() {
if rest.len()<2 {return Err(())}
let n=u16::from_be_bytes([rest[0],rest[1]]) as usize;rest=&rest[2..];
if n<2 || rest.len()<n {return Err(())}
Self::hevc_header(&rest[..n])?;
if (rest[0]>>1)&63>=48 {return Err(())}
self.append(&[0,0,0,1])?;self.append(&rest[..n])?;rest=&rest[n..];
}Ok(())
},
49 if p.len()>3=>{
let f=p[2];let kind=f&63;let start=f&128!=0;let end=f&64!=0;
if kind>=48 || (start&&end){return Err(())}
// RFC 7798 4.4.3: preserve LayerId and TID, replacing only FU's NAL type.
let header=[(p[0]&0x81)|(kind<<1),p[1]];let key=u16::from_be_bytes(header);
if start {
if self.fragment.is_some(){return Err(())}
self.append(&[0,0,0,1])?;self.append(&header)?;self.fragment=Some(key);
}else if self.fragment!=Some(key){return Err(())}
self.append(&p[3..])?;if end{self.fragment=None;}Ok(())
},
_=>Err(()),
}
}
}
/// Drop the oldest item on overflow; never block the RTP reader on a decoder/UI.
pub struct Latest<T>(Arc<Mutex<VecDeque<T>>>, usize);
@@ -136,3 +183,46 @@ mod tests {
let q=Latest::new(2);assert!(!q.push(1));q.push(2);assert!(q.push(3));assert_eq!(q.pop(),Some(2));assert_eq!(q.pop(),Some(3));
}
}
#[cfg(test)] mod hevc_tests {
use super::*;
fn packet(seq:u16,ts:u32,marker:bool,payload:Vec<u8>)->Packet {Packet{seq,timestamp:ts,marker,payload}}
fn sample()->Vec<u8>{
let mut au=vec![0,0,0,1,64,1,9,0,0,0,1,66,1,8,0,0,0,1,68,1,7,0,0,0,1,0x27,0x2b];
au.extend((0..2501).map(|i|(i%251+1)as u8));au
}
#[test] fn host_payloader_round_trip_single_fu_and_reorder(){
for mtu in [4,5,7,1190,9000] {
let au=sample();let payloads=crate::hevc::payload(mtu,&au).unwrap();let n=payloads.len();
let mut packets:Vec<_>=payloads.into_iter().enumerate().map(|(i,b)|packet(65534u16.wrapping_add(i as u16),10,i+1==n,b.to_vec())).collect();
// Start at the first sequence, then reorder adjacent packets across wraparound.
for i in (1..n.saturating_sub(1)).step_by(2){packets.swap(i,i+1);}
let mut r=Receiver::new(Codec::H265);let mut units=Vec::new();
for p in packets {let o=r.push(p);assert!(!o.pli);units.extend(o.units);}
assert_eq!(units,vec![au]);
}
}
#[test] fn aggregation_and_malformed_packets(){
let ap=vec![96,1,0,3,64,1,9,0,3,38,1,7];
let mut r=Receiver::new(Codec::H265);
assert_eq!(r.push(packet(0,0,true,ap)).units,vec![vec![0,0,0,1,64,1,9,0,0,0,1,38,1,7]]);
for payload in [vec![],vec![64],vec![128,1],vec![64,0],vec![96,1],vec![96,1,0,4,38,1,7],
vec![96,1,0,2,96,1],vec![98,1,0xc1,3],vec![98,1,0x41,3],vec![98,1,0xb0,3],vec![100,1,7]] {
let o=Receiver::new(Codec::H265).push(packet(1,1,true,payload));assert!(o.pli);assert!(o.units.is_empty());
}
}
#[test] fn loss_duplicates_incomplete_tail_and_header_mismatch(){
let payloads=crate::hevc::payload(1190,&sample()).unwrap();
let mut r=Receiver::new(Codec::H265);
// Parameter sets followed by a fragmented IDR; lose its middle fragment.
for (i,p) in payloads.iter().enumerate().take(4){r.push(packet(i as u16,1,false,p.to_vec()));}
let tail=packet(5,1,true,payloads[5].to_vec());
assert!(r.push(tail.clone()).units.is_empty());assert!(r.push(tail).units.is_empty());
let o=r.expire_at(Instant::now()+Duration::from_millis(50));assert!(o.pli);assert!(o.units.is_empty());
assert_eq!(r.push(packet(6,2,true,vec![38,1,9])).units.len(),1);
r.push(packet(7,3,false,vec![98,1,0x93,7]));
assert!(r.expire_at(Instant::now()+Duration::from_millis(201)).pli);
let mut r=Receiver::new(Codec::H265);r.push(packet(0,0,false,vec![98,1,0x93,7]));
let o=r.push(packet(1,0,true,vec![98,2,0x53,8]));assert!(o.pli);assert!(o.units.is_empty());
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ pub mod audio;
pub mod input;
pub mod desktop;
#[cfg(feature="viewer-ffmpeg")] pub mod decoder;
#[cfg(feature="viewer-sdl")] pub mod decoder_cli;
#[cfg(any(test,feature="viewer-sdl"))] pub mod decoder_cli;
#[cfg(uu_gtk)] pub mod ui;
#[cfg(uu_gtk)] pub mod tools_ui;
#[cfg(feature="viewer-sdl")] pub mod sdl;
+48 -13
View File
@@ -1,6 +1,6 @@
use super::wire::{self, Shape, CHANNELS};
use serde_json::{json, Value};
use std::sync::{Arc,atomic::{AtomicBool,AtomicU64,Ordering}};
use std::sync::{Arc,atomic::{AtomicBool,AtomicU32,AtomicU64,Ordering}};
use tokio::sync::mpsc;
use webrtc::{api::{APIBuilder,media_engine::MediaEngine,setting_engine::SettingEngine}, peer_connection::{RTCPeerConnection,configuration::RTCConfiguration,sdp::session_description::RTCSessionDescription,policy::ice_transport_policy::RTCIceTransportPolicy}, ice_transport::{ice_server::RTCIceServer,ice_candidate::RTCIceCandidateInit}, data_channel::{RTCDataChannel,data_channel_init::RTCDataChannelInit}, rtp_transceiver::{rtp_codec::RTPCodecType,rtp_transceiver_direction::RTCRtpTransceiverDirection,RTCRtpTransceiverInit}};
@@ -34,6 +34,8 @@ pub struct Observed {
pub control_messages:AtomicU64,
pub other_messages:AtomicU64,
pub media_packets:AtomicU64,
pub encoder_codecs:AtomicU32,
pub chroma:AtomicU32,
}
pub struct Peer {
clipboard:super::clipboard::Bridge,
@@ -49,17 +51,21 @@ pub struct Peer {
ice_id:String,
remote_client:String,
}
fn media_engine(shape:&Shape)->Result<MediaEngine,String>{
use webrtc::rtp_transceiver::rtp_codec::{RTCRtpCodecParameters,RTCRtpCodecCapability};
let mut engine=MediaEngine::default();
let mut codecs=vec![(RTPCodecType::Video,"video/H264",90000,0,"level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f",102),
(RTPCodecType::Audio,"audio/opus",48000,2,"minptime=10;useinbandfec=1",111)];
// RFC 7798 single-layer non-interleaved mode: no sprop-max-don-diff, hence no DONL.
if shape.decoders.iter().any(|d|d.codec==2){codecs.insert(0,(RTPCodecType::Video,"video/H265",90000,0,"",104));}
for (kind,mime,rate,channels,fmtp,payload) in codecs {
engine.register_codec(RTCRtpCodecParameters{capability:RTCRtpCodecCapability{mime_type:mime.into(),clock_rate:rate,channels,sdp_fmtp_line:fmtp.into(),..Default::default()},payload_type:payload,..Default::default()},kind).map_err(|_|"Codec initialization failed")?;
}Ok(engine)
}
impl Peer {
pub async fn new(config:RTCConfiguration, shape:&Shape, app_id:&str, ice_id:&str, remote_client:&str, outbound:mpsc::Sender<Value>, settings:SettingEngine)->Result<Self,String> {
shape.validate_session()?;
let mut engine=MediaEngine::default();
use webrtc::rtp_transceiver::rtp_codec::{RTCRtpCodecParameters,RTCRtpCodecCapability};
for (kind,mime,rate,channels,fmtp,payload) in [
(RTPCodecType::Video,"video/H264",90000,0,"level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f",102),
(RTPCodecType::Audio,"audio/opus",48000,2,"minptime=10;useinbandfec=1",111),
] {
engine.register_codec(RTCRtpCodecParameters{capability:RTCRtpCodecCapability{mime_type:mime.into(),clock_rate:rate,channels,sdp_fmtp_line:fmtp.into(),..Default::default()},payload_type:payload,..Default::default()},kind).map_err(|_|"Codec initialization failed")?;
}
let mut engine=media_engine(shape)?;
// UU RTP metadata the official Mac viewer offers (src/rtp_metadata.rs on the host side).
for uri in ["http://www.webrtc.org/experiments/rtp-hdrext/video-capture-index","http://www.webrtc.org/experiments/rtp-hdrext/video-frame-is-new-frame"] {
engine.register_header_extension(webrtc::rtp_transceiver::rtp_codec::RTCRtpHeaderExtensionCapability{uri:uri.into()},RTPCodecType::Video,None).map_err(|_|"RTP header extension registration failed")?;
@@ -104,9 +110,9 @@ impl Peer {
let video=track.kind()==RTPCodecType::Video;
log::info!("Viewer track: kind={} mime={} payload_type={}",track.kind(),track.codec().capability.mime_type,track.payload_type());
let opus=track.kind()==RTPCodecType::Audio && track.codec().capability.mime_type.eq_ignore_ascii_case("audio/opus");
if video && !track.codec().capability.mime_type.eq_ignore_ascii_case("video/H264") {return}
if video && super::media::Codec::from_mime(&track.codec().capability.mime_type).is_none() {return}
if video {desktop.ssrc.store(track.ssrc(),Ordering::Release);}
let mut receiver=super::media::Receiver::default();
let mut receiver=super::media::Receiver::new(super::media::Codec::from_mime(&track.codec().capability.mime_type).unwrap_or_default());
let mut tick=tokio::time::interval(std::time::Duration::from_millis(40));
let mut last_pli=std::time::Instant::now()-std::time::Duration::from_secs(1);
loop {
@@ -115,6 +121,12 @@ impl Peer {
Ok((p,_))=>{
counters.media_packets.fetch_add(1,Ordering::Relaxed);
if !video {if opus {audio.push(p.header.sequence_number,&p.payload);}continue}
// TrackRemote updates codec from the negotiated payload type on each read.
let Some(codec)=super::media::Codec::from_mime(&track.codec().capability.mime_type) else{continue};
if receiver.codec()!=codec {
receiver=super::media::Receiver::new(codec);
desktop.reset.store(true,Ordering::Release);desktop.pli.store(true,Ordering::Release);
}
receiver.push(super::media::Packet{seq:p.header.sequence_number,timestamp:p.header.timestamp,marker:p.header.marker,payload:p.payload.to_vec()})
},
Err(_)=>break,
@@ -122,7 +134,7 @@ impl Peer {
_=tick.tick()=>{if !video {continue} receiver.expire()},
};
if output.pli {desktop.reset.store(true,Ordering::Release);desktop.pli.store(true,Ordering::Release);}
for au in output.units {if desktop.units.push(au) {desktop.reset.store(true,Ordering::Release);desktop.pli.store(true,Ordering::Release);}}
for au in output.units {if desktop.units.push(super::media::AccessUnit{codec:receiver.codec(),data:au}) {desktop.reset.store(true,Ordering::Release);desktop.pli.store(true,Ordering::Release);}}
if desktop.pli.load(Ordering::Acquire) && last_pli.elapsed()>=std::time::Duration::from_millis(250) {
desktop.pli.swap(false,Ordering::AcqRel);
if let Some(pc)=weak_pc.upgrade() {
@@ -160,9 +172,13 @@ impl Peer {
for (index,channel) in channels.iter().enumerate() {
if index==0 {
let weak=Arc::downgrade(&channel);let state=observed.clone();
let chroma444=shape.capture_type==1 && std::env::var("UU_VIEWER_CHROMA").is_ok_and(|v|v=="444");
channel.on_open(Box::new(move ||Box::pin(async move {
state.control_open.store(true,Ordering::Release);
if let Some(channel)=weak.upgrade(){let _=channel.send(&bytes::Bytes::from(wire::echo_request_with_clipboard(1,clipboard_enabled))).await;}
if let Some(channel)=weak.upgrade(){
let _=channel.send(&bytes::Bytes::from(wire::echo_request_with_clipboard(1,clipboard_enabled))).await;
if chroma444 {let _=channel.send(&bytes::Bytes::from(wire::request_chroma444())).await;}
}
})));
}
let closed=tools.clone();let state=observed.clone();
@@ -174,6 +190,11 @@ impl Peer {
let control_for_echo=if index==0 {Arc::downgrade(&channel)} else {std::sync::Weak::new()};
channel.on_message(Box::new(move |message| {let state=state.clone();let inbox=inbox.clone();let control_for_echo=control_for_echo.clone();let clipboard_tx=clipboard_tx.clone();Box::pin(async move {
if index==0 {
let (codecs,chroma)=wire::media_control(&message.data);
if let Some(codecs)=codecs {state.encoder_codecs.store(codecs,Ordering::Release);}
if let Some(chroma)=chroma {
if state.chroma.swap(chroma,Ordering::AcqRel)!=chroma {log::info!("Viewer capture sync: chroma={chroma}");}
}
let count=state.control_messages.fetch_add(1,Ordering::Relaxed);
if wire::echo_matches(&message.data,1) {state.clipboard_supported.store(wire::echo_clipboard_supported(&message.data),Ordering::Release);state.echo_received.store(true,Ordering::Release);}
if let Some(request)=wire::echo_request_seq(&message.data) {
@@ -313,3 +334,17 @@ pub fn with_max_message_size(sdp:&str)->String{
assert_eq!(with_max_message_size("m=application 9\r\na=sctp-port:5000\r\na=mid:0\r\n"),"m=application 9\r\na=sctp-port:5000\r\na=max-message-size:262144\r\na=mid:0\r\n");
assert_eq!(with_max_message_size("a=max-message-size:1\r\n"),"a=max-message-size:1\r\n");
}
#[cfg(test)] #[tokio::test] async fn offers_both_codecs_or_h264_only(){
for force in [false,true] {
let shape:Shape=serde_json::from_str(include_str!("../../tests/fixtures/viewer/session-shape.json")).unwrap();
let shape=shape.video_preferences(force);
let api=APIBuilder::new().with_media_engine(media_engine(&shape).unwrap()).build();
let pc=api.new_peer_connection(RTCConfiguration::default()).await.unwrap();
pc.add_transceiver_from_kind(RTPCodecType::Video,Some(RTCRtpTransceiverInit{direction:RTCRtpTransceiverDirection::Recvonly,send_encodings:vec![]})).await.unwrap();
let offer=pc.create_offer(None).await.unwrap();
assert!(offer.sdp.contains("a=rtpmap:102 H264/90000"));
assert_eq!(offer.sdp.contains("a=rtpmap:104 H265/90000"),!force);
assert!(!offer.sdp.contains("sprop-max-don-diff"));pc.close().await.unwrap();
}
}
+6 -4
View File
@@ -1,13 +1,14 @@
use super::{desktop::{Frame,FrameSink},input::{Input,Viewport},session::Session};
use sdl2::{event::{Event,WindowEvent},mouse::MouseButton,pixels::PixelFormatEnum,rect::Rect};
pub struct Window { canvas:sdl2::render::Canvas<sdl2::video::Window>, events:sdl2::EventPump, _sdl:sdl2::Sdl, frame:(u32,u32), focused:bool, files:bool }
pub struct Window { canvas:sdl2::render::Canvas<sdl2::video::Window>, events:sdl2::EventPump, _sdl:sdl2::Sdl, frame:(u32,u32), focused:bool, files:bool, mac:bool }
impl Window {
pub fn new()->Result<Self,String>{
let sdl=sdl2::init()?;let video=sdl.video()?;
let window=video.window("Native viewer (experimental)",1280,720).position_centered().resizable().build().map_err(|_|"Window creation failed")?;
let canvas=window.into_canvas().build().map_err(|_|"Renderer creation failed")?;
let events=sdl.event_pump()?;Ok(Self{canvas,events,_sdl:sdl,frame:(1920,1080),focused:true,files:false})
let events=sdl.event_pump()?;Ok(Self{canvas,events,_sdl:sdl,frame:(1920,1080),focused:true,files:false,mac:false})
}
pub fn set_mac(&mut self,mac:bool){self.mac=mac;}
fn viewport(&self)->Viewport {Viewport::fit(self.canvas.window().size(),self.frame)}
pub fn poll(&mut self,input:&mut Input)->(bool,Vec<serde_json::Value>){
let viewport=self.viewport();
@@ -67,7 +68,7 @@ pub async fn run(session:&mut Session,window:&mut Window,decoder:&mut dyn super:
let mut command_open=true;let mut requested=false;
let mut terminate=tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate()).map_err(|_|"Cannot watch termination")?;
let mut audio=super::audio::Playback::new(&window._sdl,session.peer.audio.clone());
let mut input=Input::new(view_only);let mut tick=tokio::time::interval(std::time::Duration::from_millis(8));
let mut input=Input::new(view_only);input.set_mac(window.mac);let mut tick=tokio::time::interval(std::time::Duration::from_millis(8));
let mut heartbeat=tokio::time::interval(std::time::Duration::from_secs(1));
let deadline=tokio::time::sleep(std::time::Duration::from_secs(30));tokio::pin!(deadline);
let result=async {
@@ -106,7 +107,8 @@ pub async fn run(session:&mut Session,window:&mut Window,decoder:&mut dyn super:
// At most one AU per iteration keeps local event polling responsive.
if let Some(au)=session.peer.desktop.units.pop(){
if session.peer.desktop.reset.swap(false,std::sync::atomic::Ordering::AcqRel){decoder.reset();}
super::desktop::display(decoder,window,&au,&session.peer.desktop.pli)?;}
decoder.set_codec(au.codec)?;
super::desktop::display(decoder,window,&au.data,au.codec,&session.peer.desktop.pli)?;}
}
}}
}.await;
+3 -1
View File
@@ -8,6 +8,8 @@ pub struct Options {pub timeout:Duration,pub legacy_prefix:bool,pub ready_policy
pub struct Session {pub peer:Peer,signal:Signal,candidates:mpsc::Receiver<Value>,policy:ReadyPolicy}
impl Session {
pub async fn open(room:&JoinData,shape:&Shape,device:&str,client:&str,options:Options,settings:SettingEngine)->Result<Self,String> {
shape.validate_session()?;
let shape=shape.video_preferences(std::env::var("UU_VIEWER_CODEC").is_ok_and(|v|v.eq_ignore_ascii_case("h264")));
shape.validate_session()?;
let app=wire::uuid();let ice=hex::encode(rand::random::<[u8;16]>());
let timeout=if room.ws_connect_timeout_ms>0 {Duration::from_millis(room.ws_connect_timeout_ms as u64)}else{options.timeout};
@@ -31,7 +33,7 @@ impl Session {
let remote=result.get("client_id").and_then(Value::as_str).filter(|s|!s.is_empty()).ok_or("Control acknowledgement missing peer route")?;
let config=rtc::configuration(&result)?;
let(tx,rx)=mpsc::channel(256);
let peer=Peer::new(config,shape,&app,&ice,remote,tx,settings).await?;
let peer=Peer::new(config,&shape,&app,&ice,remote,tx,settings).await?;
// The official host drops an offer whose ice_id is not yet in its connection table
// (libstreamer 0xc3080); give it time to process be-controlled first.
let delay=std::env::var("UU_VIEWER_OFFER_DELAY_MS").ok().and_then(|v|v.parse().ok()).unwrap_or(0u64).min(10_000);
+56 -4
View File
@@ -130,7 +130,7 @@ async fn complete_session_uses_loopback_signaling_and_virtual_rtc_through_echo()
Some("offer")=> {
let offered=body["sdp"].as_str().unwrap();
assert!(offered.contains("H264/90000") && offered.contains("opus/48000"));
assert!(!offered.contains("H265") && !offered.contains("VP8"));
assert!(offered.contains("H265/90000") && !offered.contains("VP8"));
assert_eq!(offered.matches("a=recvonly").count(),2);
server_host.set_remote_description(RTCSessionDescription::offer(offered.into()).unwrap()).await.unwrap();
let answer=server_host.create_answer(None).await.unwrap();let sdp=answer.sdp.clone();let mut gathered=server_host.gathering_complete_promise().await;server_host.set_local_description(answer).await.unwrap();
@@ -169,7 +169,7 @@ async fn complete_session_uses_loopback_signaling_and_virtual_rtc_through_echo()
let heartbeat=super::input::Input::heartbeat();session.peer.send_input(&heartbeat).await.unwrap();assert_eq!(input_rx.recv().await.unwrap(),heartbeat);
let packet=|seq,ts,marker,payload:&[u8]|webrtc::rtp::packet::Packet{header:webrtc::rtp::header::Header{version:2,sequence_number:seq,timestamp:ts,marker,..Default::default()},payload:bytes::Bytes::copy_from_slice(payload)};
track.write_rtp(&packet(9,1,true,&[0x65,1])).await.unwrap();
loop {if let Some(au)=session.peer.desktop.units.pop(){assert_eq!(au,vec![0,0,0,1,0x65,1]);break}tokio::time::sleep(Duration::from_millis(10)).await;}
loop {if let Some(au)=session.peer.desktop.units.pop(){assert_eq!(au.codec,super::media::Codec::H264);assert_eq!(au.data,vec![0,0,0,1,0x65,1]);break}tokio::time::sleep(Duration::from_millis(10)).await;}
track.write_rtp(&packet(10,2,false,&[0x7c,0x85,1])).await.unwrap();
track.write_rtp(&packet(12,2,true,&[0x7c,0x45,3])).await.unwrap();
loop {let (packets,_)=sender.read_rtcp().await.unwrap();if packets.iter().any(|p|p.as_any().is::<webrtc::rtcp::payload_feedbacks::picture_loss_indication::PictureLossIndication>()){break}}
@@ -220,7 +220,7 @@ async fn opus_and_video_tracks_are_received_independently_in_memory() {
let video=Arc::new(TrackLocalStaticRTP::new(RTCRtpCodecCapability{mime_type:"video/H264".into(),clock_rate:90000,sdp_fmtp_line:"level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f".into(),..Default::default()},"video".into(),"test".into()));
host.add_track(audio.clone() as Arc<dyn TrackLocal+Send+Sync>).await.unwrap();host.add_track(video.clone() as Arc<dyn TrackLocal+Send+Sync>).await.unwrap();
let(tx,_rx)=mpsc::channel(256);
let peer=Peer::new(RTCConfiguration::default(),&shape(),"app","ice","host",tx,viewer_settings).await.unwrap();
let peer=Peer::new(RTCConfiguration::default(),&shape().video_preferences(false),"app","ice","host",tx,viewer_settings).await.unwrap();
let mut gathered=peer.pc.gathering_complete_promise().await;peer.offer().await.unwrap();gathered.recv().await;
let offer=peer.pc.local_description().await.unwrap();assert!(offer.sdp.contains("m=audio") && offer.sdp.contains("opus/48000/2"));
host.set_remote_description(offer).await.unwrap();let answer=host.create_answer(None).await.unwrap();
@@ -233,7 +233,7 @@ async fn opus_and_video_tracks_are_received_independently_in_memory() {
audio.write_rtp(&packet(&[0xf8,0xff,0xfe])).await.unwrap();video.write_rtp(&packet(&[0x65,1])).await.unwrap();
tokio::time::sleep(Duration::from_millis(10)).await;
if let Some(data)=peer.audio.pop(){assert_eq!(data,vec![0xf8,0xff,0xfe]);heard=true;}
if let Some(data)=peer.desktop.units.pop(){assert_eq!(data,vec![0,0,0,1,0x65,1]);seen=true;}
if let Some(data)=peer.desktop.units.pop(){assert_eq!(data.codec,super::media::Codec::H264);assert_eq!(data.data,vec![0,0,0,1,0x65,1]);seen=true;}
if heard && seen {break}
}
assert!(heard && seen);peer.close().await;host.close().await.unwrap();router.lock().await.stop().await.unwrap();
@@ -287,3 +287,55 @@ async fn desktop_file_window_reattaches_without_replacing_peer(){
peer.close().await;host.close().await.unwrap();router.lock().await.stop().await.unwrap();
}).await.expect("desktop file reuse timed out");
}
#[tokio::test]
async fn hevc_track_uses_negotiated_payload_type_in_memory(){
tokio::time::timeout(Duration::from_secs(10),async {
use super::media::Codec;
use webrtc::{track::track_local::{track_local_static_rtp::TrackLocalStaticRTP,TrackLocal,TrackLocalWriter},
rtp_transceiver::rtp_codec::{RTCRtpCodecCapability,RTCRtpCodecParameters,RTPCodecType}};
let(viewer_settings,host_settings,router)=virtual_network().await;
// The host uses a different local PT; the answer binds its track to the viewer offer.
let mut engine=MediaEngine::default();
let capability=RTCRtpCodecCapability{mime_type:"video/H265".into(),clock_rate:90000,..Default::default()};
engine.register_codec(RTCRtpCodecParameters{capability:capability.clone(),payload_type:126,..Default::default()},RTPCodecType::Video).unwrap();
let host=Arc::new(APIBuilder::new().with_media_engine(engine).with_setting_engine(host_settings).build().new_peer_connection(RTCConfiguration::default()).await.unwrap());
let video=Arc::new(TrackLocalStaticRTP::new(capability,"video".into(),"test".into()));
host.add_track(video.clone() as Arc<dyn TrackLocal+Send+Sync>).await.unwrap();
host.on_data_channel(Box::new(|channel|Box::pin(async move {
if channel.label()=="CONTROL_DATA_CHANNEL" {
let weak=Arc::downgrade(&channel);
channel.on_open(Box::new(move||Box::pin(async move {
if let Some(channel)=weak.upgrade(){
let mut caps=Vec::new();for codec in [1,2]{let mut cap=Vec::new();wire::int(3,codec,&mut cap);wire::bytes(1,&cap,&mut caps);}
channel.send(&bytes::Bytes::from(wire::envelope(2,18,&caps))).await.unwrap();
// Model a host that declines 4:4:4 and encodes 4:2:0.
channel.send(&bytes::Bytes::from(wire::envelope(3,25,&[24,1]))).await.unwrap();
}
})));
}
})));
let(tx,_rx)=mpsc::channel(256);
let peer=Peer::new(RTCConfiguration::default(),&shape().video_preferences(false),"app","ice","host",tx,viewer_settings).await.unwrap();
let mut gathered=peer.pc.gathering_complete_promise().await;peer.offer().await.unwrap();gathered.recv().await;
host.set_remote_description(peer.pc.local_description().await.unwrap()).await.unwrap();
let answer=host.create_answer(None).await.unwrap();
assert!(answer.sdp.contains("H265/90000"));assert!(!answer.sdp.contains("H264/90000"));
let mut gathered=host.gathering_complete_promise().await;host.set_local_description(answer).await.unwrap();gathered.recv().await;
peer.pc.set_remote_description(host.local_description().await.unwrap()).await.unwrap();
while peer.pc.connection_state()!=webrtc::peer_connection::peer_connection_state::RTCPeerConnectionState::Connected {tokio::time::sleep(Duration::from_millis(10)).await;}
let mut au=vec![0,0,0,1,64,1,9,0,0,0,1,38,1];au.extend(vec![7;200]);
let payloads=crate::hevc::payload(80,&au).unwrap();let mut seq=0;let mut seen=false;
for timestamp in 0..100 {
for (i,payload) in payloads.iter().enumerate(){
video.write_rtp(&webrtc::rtp::packet::Packet{header:webrtc::rtp::header::Header{version:2,sequence_number:seq,timestamp:timestamp*3000,marker:i+1==payloads.len(),..Default::default()},payload:payload.clone()}).await.unwrap();seq=seq.wrapping_add(1);
}
tokio::time::sleep(Duration::from_millis(10)).await;
if let Some(unit)=peer.desktop.units.pop(){assert_eq!(unit.codec,Codec::H265);assert_eq!(unit.data,au);seen=true;}
if seen && peer.observed.chroma.load(Ordering::Acquire)==1 {break}
}
assert!(seen);assert_eq!(peer.observed.encoder_codecs.load(Ordering::Acquire),3);
assert_eq!(peer.observed.chroma.load(Ordering::Acquire),1);
peer.close().await;host.close().await.unwrap();router.lock().await.stop().await.unwrap();
}).await.expect("in-memory H265 timed out");
}
+39 -4
View File
@@ -16,6 +16,19 @@ pub fn inflate(data:&[u8])->Result<Vec<u8>,String>{
let mut out=Vec::new();flate2::read::ZlibDecoder::new(data).take(16*1024*1024+1).read_to_end(&mut out).map_err(|_|"Invalid directory compression")?;
if out.len()>16*1024*1024{return Err("Directory expansion limit".into())}Ok(out)
}
/// Splits a remote target into (directory, file name). Windows hosts list paths with `\\` (C:\\Users\\...),
/// Linux hosts with `/`; the directory keeps the host's own separator style. No directory means home.
/// Download block limit. Our host sends 32 KiB blocks; the official Windows host sends larger ones
/// (a 64 KiB file arrived as one block), bounded by the 256 KiB SCTP message size.
const MAX_BLOCK:usize=256*1024;
fn split_remote(remote:&str)->Option<(&str,&str)>{
let cut=remote.rfind(['/','\\']);
let (dir,name)=match cut{Some(i)=>(&remote[..i],&remote[i+1..]),None=>("",remote)};
if name.is_empty(){return None}
// "/x" keeps the root; "C:\\x" keeps "C:\\" (a bare drive letter is not a directory).
let dir=match cut{None=>"~/",Some(i) if dir.is_empty()||dir.ends_with(':')=>&remote[..i+1],_=>dir};
Some((dir,name))
}
fn safe(path:&str)->Result<PathBuf,String>{
let normalized=path.replace('\\',"/");let p=Path::new(&normalized);
if p.as_os_str().is_empty() || p.components().any(|c|!matches!(c,std::path::Component::Normal(_))) || normalized.contains(':') {return Err("Unsafe remote relative path".into())}Ok(p.to_owned())
@@ -58,11 +71,22 @@ impl Files {
let mut rx=self.rx.lock().await;self.canceled.store(false,Ordering::Release);
let mut file=tokio::fs::File::open(local).await.map_err(|_|"Cannot open upload file")?;
let meta=file.metadata().await.map_err(|_|"Cannot stat upload file")?;if !meta.is_file(){return Err("Upload requires a regular file".into())}let size=meta.len();
let remote=Path::new(remote);let name=remote.file_name().and_then(|s|s.to_str()).ok_or("Invalid remote file name")?;safe(name)?;
let parent=remote.parent().and_then(|p|p.to_str()).filter(|p|!p.is_empty()).unwrap_or("~/");
let (parent,name)=split_remote(remote).ok_or("Invalid remote file name")?;safe(name)?;
let task=self.next.fetch_add(1,Ordering::Relaxed);
let mut info=nested(1,name.as_bytes());wire::int(2,size,&mut info);
// GvPb_FileInfo {1 rel_path, 2 size, 3 modified_time (unix s)}; receive_request {.., 7 task_unique_id}.
// The official Windows host rejects (code 2) a request without the modified time / unique id.
let modified=meta.modified().ok().and_then(|t|t.duration_since(std::time::UNIX_EPOCH).ok()).map_or(0,|d|d.as_secs());
let mut info=nested(1,name.as_bytes());wire::int(2,size,&mut info);wire::int(3,modified,&mut info);
let now=std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).unwrap_or_default();
let unique=format!("{:08x}-{:04x}-4{:03x}-8{:03x}-{:012x}",now.as_secs() as u32,task&0xffff,(now.subsec_nanos()>>20)&0xfff,std::process::id()&0xfff,now.as_nanos()&0xffff_ffff_ffff);
let mut body=id(task,0);wire::bytes(2,parent.as_bytes(),&mut body);wire::bytes(3,&info,&mut body);wire::int(4,1,&mut body);
// Official viewers with the list-compress feature send FileList{1 files} zlib-compressed in list_data (6);
// the official Windows host only registers files from there. Keep the inline copy for older hosts.
let mut list=Vec::new();wire::bytes(1,&info,&mut list);
let mut z=flate2::write::ZlibEncoder::new(Vec::new(),flate2::Compression::default());
std::io::Write::write_all(&mut z,&list).map_err(|_|"Cannot compress file list")?;
wire::bytes(6,&z.finish().map_err(|_|"Cannot compress file list")?,&mut body);
wire::bytes(7,unique.as_bytes(),&mut body);
let result=async {
let r=self.request(&mut rx,1,&body,1).await?;check(uint(&r,3))?;
let mut ask=id(task,1);wire::int(3,size,&mut ask);let r=self.request(&mut rx,3,&ask,3).await?;check(uint(&r,3))?;
@@ -120,7 +144,7 @@ impl Files {
let block=uint(body,2);let chunk=blob(body,3).unwrap_or_default();
if let Some(d)=open.get_mut(&index){
let expected=files[index as usize-1].1;
if chunk.len()>32*1024 || d.next.is_some_and(|n|block<n) || d.blocks.contains_key(&block)||d.blocks.len()>=32 {code=3}else{
if chunk.len()>MAX_BLOCK || d.next.is_some_and(|n|block<n) || d.blocks.contains_key(&block)||d.blocks.len()>=32 {code=3}else{
d.blocks.insert(block,chunk.to_vec());
if d.next.is_none() && (block==0 || d.blocks.len()>=16) {d.next=d.blocks.keys().next().copied();}
while let Some(chunk)=d.next.and_then(|n|d.blocks.remove(&n)){
@@ -176,3 +200,14 @@ impl Files {
result
}
}
#[cfg(test)]
mod split_tests{
#[test] fn remote_targets_keep_the_host_separator(){
assert_eq!(super::split_remote(r"C:\Users\Public\Downloads\a.bin"),Some((r"C:\Users\Public\Downloads","a.bin")));
assert_eq!(super::split_remote(r"C:\a.bin"),Some((r"C:\","a.bin")));
assert_eq!(super::split_remote("/tmp/a.bin"),Some(("/tmp","a.bin")));
assert_eq!(super::split_remote("/a.bin"),Some(("/","a.bin")));
assert_eq!(super::split_remote("a.bin"),Some(("~/","a.bin")));
assert_eq!(super::split_remote("/tmp/"),None);
}
}
+14 -4
View File
@@ -13,13 +13,21 @@ impl Drop for Route {fn drop(&mut self){self.routes.lock().unwrap().remove(&self
impl Drop for PortMap {fn drop(&mut self){self.worker.abort();for(_,r)in self.rules.lock().unwrap().drain(){let _=r.stop.send(true);}self.routes.lock().unwrap().clear();}}
impl PortMap {
pub fn new(io:Transport,mut rx:mpsc::Receiver<Packet>)->Self{
let session=rand::random::<u64>();let routes:Routes=Arc::new(Mutex::new(HashMap::new()));let route=routes.clone();
// The official Windows host answers every frame with session_id 1 whatever the viewer sent (observed),
// and this project's host echoes it; one port-mapping peer carries one session, so use 1.
let session=1u64;let routes:Routes=Arc::new(Mutex::new(HashMap::new()));let route=routes.clone();
let(life,alive)=watch::channel(());
let worker=tokio::spawn(async move{
let _life=life;
while let Some((channel,b))=rx.recv().await {
if channel!=FILE{continue}let Some(f)=blob(&b,27)else{continue};if uint(f,1)!=session{continue}
if channel!=FILE{continue}let Some(f)=blob(&b,27)else{continue};
let key=(uint(f,2),uint(f,3));let sender=route.lock().unwrap().get(&key).cloned();
// Diagnostics without payload: wire types per field and whether session/route matched.
if uint(f,1)!=session || sender.is_none() {
let kinds=crate::protocol::decode_wire(f).map(|x|x.iter().map(|w|(w.number,match w.value{crate::protocol::WireValue::Varint(_)=>"varint",crate::protocol::WireValue::Bytes(_)=>"bytes",_=>"fixed"})).collect::<Vec<_>>()).unwrap_or_default();
log::warn!("Port mapping frame unmatched: fields={kinds:?} session_match={} route_match={} type={}",uint(f,1)==session,sender.is_some(),uint(f,4));
continue;
}
if let Some(tx)=sender {if tx.try_send((uint(f,4),blob(f,5).unwrap_or_default().to_vec())).is_err(){route.lock().unwrap().remove(&key);}}
}
route.lock().unwrap().clear();
@@ -51,7 +59,8 @@ impl PortMap {
let io=io.clone();let host=host.clone();let mut cancel=stop.clone();let mut transport_closed=disconnected.clone();
streams.spawn(async move {
let result=tokio::select!{r=forward(&io,session,id,stream,socket,rx,&host,target_port)=>r,_=cancel.changed()=>Ok(()),_=transport_closed.changed()=>Err("Forwarding transport closed".into())};
if result.is_err(){guard.status.lock().unwrap().failures+=1;}
// Reason only (fixed strings), never forwarded bytes or the target address.
if let Err(reason)=&result{guard.status.lock().unwrap().failures+=1;log::warn!("Port mapping stream ended: {reason}");}
let _=io.send(FILE,frame(session,id,stream,1,&[])).await;drop(guard);
});
}
@@ -71,7 +80,8 @@ async fn forward(io:&Transport,session:u64,rule:u64,stream:u64,mut socket:TcpStr
let syn=serde_json::json!({"version":1,"target_host":host,"target_port":port}).to_string();
io.send(FILE,frame(session,rule,stream,0,syn.as_bytes())).await?;
let ack=tokio::time::timeout(Duration::from_secs(12),rx.recv()).await.map_err(|_|"Forwarding SYN timeout")?.ok_or("Forwarding channel closed")?;
if ack.0!=3 || serde_json::from_slice::<serde_json::Value>(&ack.1).ok().and_then(|v|v["ok"].as_bool())!=Some(true){return Err("Forwarding target rejected".into())}
if ack.0!=3 {return Err(format!("Forwarding SYN answered with frame type {}",ack.0))}
if serde_json::from_slice::<serde_json::Value>(&ack.1).ok().and_then(|v|v["ok"].as_bool())!=Some(true){return Err("Forwarding target rejected".into())}
let mut buf=vec![0;16*1024];
loop {tokio::select!{
r=socket.read(&mut buf)=>{let n=r.map_err(|_|"Forwarding read failed")?;if n==0{break}io.send(FILE,frame(session,rule,stream,2,&buf[..n])).await?;},
+18 -6
View File
@@ -57,7 +57,9 @@ impl Terminal {
if let Some(event)=event {if tx.try_send(event).is_err(){overflow=true;break}}
let key=if f.kind==OPEN_RESP {(OPEN_RESP,0)}else{(f.kind,f.session as u64)};
let payload=if f.kind==OPEN_RESP {
serde_json::from_slice::<Value>(&f.payload).ok().filter(Value::is_object).map(|mut v|{v["session_id"]=json!(f.session);v.to_string().into_bytes()}).unwrap_or(f.payload)
// Our host puts the new session in the frame header; the official Windows host answers with header 0,
// so only a nonzero header overrides the JSON body.
serde_json::from_slice::<Value>(&f.payload).ok().filter(Value::is_object).map(|mut v|{if f.session!=0{v["session_id"]=json!(f.session);}v.to_string().into_bytes()}).unwrap_or(f.payload)
}else{f.payload};
if let Some(t)=p.lock().unwrap().remove(&key){let _=t.send(Ok(payload));}
}
@@ -79,8 +81,12 @@ impl Terminal {
}
pub async fn check_env(&self)->Result<(),String>{
let id=self.rpc.fetch_add(1,Ordering::Relaxed);let mut r=nested(1,&ints(&[(1,id)]));wire::bytes(28,&[],&mut r);
let b=self.request((24,id),CONTROL,wire::envelope(id,21,&r)).await?;
if uint(&b,1)!=0 {return Err("Terminal environment unavailable".into())}Ok(())
// The official Windows host's rpc_main.proto has no check_env (requests end at 25): no answer means
// "not implemented", not "unavailable"; go on and let OPEN report availability.
match tokio::time::timeout(std::time::Duration::from_secs(3),self.request((24,id),CONTROL,wire::envelope(id,21,&r))).await{
Err(_)=>Ok(()),
Ok(b)=>{let b=b?;if uint(&b,1)!=0 {return Err("Terminal environment unavailable".into())}Ok(())}
}
}
pub async fn list_sessions(&self)->Result<Vec<Value>,String>{
let id=self.next.fetch_add(1,Ordering::Relaxed);
@@ -94,10 +100,16 @@ impl Terminal {
self.open_session(id,cols,rows).await
}
async fn open_session(&self,id:u32,cols:u16,rows:u16)->Result<u32,String>{
// Host TerminalConn handles OPEN(nonzero) as reattachment.
let b=self.request((OPEN_RESP,0),BINARY,encode(OPEN,id,json!({"cols":cols.max(1),"rows":rows.max(1)}).to_string().as_bytes())).await?;
// Host TerminalConn handles OPEN(nonzero) as reattachment. The official Windows host rejects an OPEN
// without "option" ("terminal open: missing 'option' field", kError 1004); "none" is its plain mode.
let b=self.request((OPEN_RESP,0),BINARY,encode(OPEN,id,json!({"cols":cols.max(1),"rows":rows.max(1),"option":"none"}).to_string().as_bytes())).await?;
let v:Value=serde_json::from_slice(&b).map_err(|_|"Invalid terminal open response")?;
if v["code"].as_i64()!=Some(0){return Err("Terminal open rejected".into())}v["session_id"].as_u64().and_then(|n|u32::try_from(n).ok()).filter(|n|*n!=0).ok_or("Invalid terminal session id".into())
if v["code"].as_i64()!=Some(0){return Err("Terminal open rejected".into())}
let id=v["session_id"].as_u64().or_else(||v["session_id"].as_str().and_then(|s|s.parse().ok())).and_then(|n|u32::try_from(n).ok()).filter(|n|*n!=0);
// The official Windows host answers {"code":0,"message"} with header 0 and keeps one terminal per
// connection addressed as session 0 (it then prints "[terminal] resumed existing session").
if id.is_none(){log::info!("Terminal open response without a session id: keys={:?}; using session 0",v.as_object().map(|o|o.keys().collect::<Vec<_>>()));}
Ok(id.unwrap_or(0))
}
pub async fn write(&self,id:u32,data:&[u8])->Result<(),String>{for b in data.chunks(16*1024){self.io.send(BINARY,encode(STDIN,id,b)).await?}Ok(())}
pub async fn resize(&self,id:u32,cols:u16,rows:u16)->Result<(),String>{self.io.send(BINARY,encode(RESIZE,id,json!({"cols":cols.max(1),"rows":rows.max(1)}).to_string().as_bytes())).await}
+73 -6
View File
@@ -52,14 +52,29 @@ impl Shape {
}
pub fn validate_session(&self)->Result<(),String> {
self.validate()?;
// H264/420 receive only. streamer_data video_codec_capability is sent by the official Mac
// viewer (8 entries observed live), so H264/420 entries are allowed there too.
if self.decoders.iter().any(|d|d.codec!=1 || d.chroma!=1) || self.codecs.iter().any(|c|c.video_codec!=1 || c.chroma_sampling!=1)
|| (self.capture_type==1 && (self.chroma!=1 || self.decoders.is_empty())) {
return Err("Sessions require H264/420 decoder and codec capabilities".into());
if self.codecs.iter().any(|c|![1,2].contains(&c.video_codec) || ![1,3].contains(&c.chroma_sampling))
|| (self.capture_type==1 && self.decoders.is_empty()) {
return Err("Sessions require supported decoder and codec capabilities".into());
}
Ok(())
}
/// Advertise software decoding of both 8-bit chroma formats. Keep caller size/fps limits.
/// The explicit chroma request is sent on CONTROL, so initial capture stays 4:2:0.
pub fn video_preferences(&self,h264_only:bool)->Self {
let mut shape=self.clone();
if shape.capture_type==1 {
shape.chroma=1;
shape.decoders.clear();
for codec in if h264_only {&[1][..]}else{&[2,1][..]} {
let base=self.decoders.iter().find(|d|d.codec==*codec).or_else(||self.decoders.first());
if let Some(base)=base {for chroma in [1,3] {
let mut d=base.clone();d.codec=*codec;d.chroma=chroma;shape.decoders.push(d);
}}
}
}
if h264_only {shape.decoders.retain(|d|d.codec==1);shape.codecs.retain(|c|c.video_codec==1);}
shape
}
pub fn connect_options(&self, device:&str) -> Result<Vec<u8>,String> {
self.validate()?;
let mut out=Vec::new(); int(1,self.capture_type.into(),&mut out);
@@ -128,7 +143,8 @@ pub fn echo_request_seq(data:&[u8]) -> Option<u64> {
let fields=decode_wire(data).ok()?;
let seq=fields.iter().find_map(|f|match (f.number,&f.value){(1,WireValue::Varint(v))=>Some(*v),_=>None})?;
fields.iter().find_map(|f|match (f.number,&f.value) {
(3,WireValue::Bytes(action))=>decode_wire(action).ok()?.iter().any(|f|f.number==1 && f.value==WireValue::Varint(0)).then_some(seq),
// proto3 omits the default action 0, so a SimpleAction without field 1 is an echo request too.
(3,WireValue::Bytes(action))=>decode_wire(action).ok()?.iter().find_map(|f|match (f.number,&f.value){(1,WireValue::Varint(v))=>Some(*v),_=>None}).unwrap_or(0).eq(&0).then_some(seq),
_=>None,
})
}
@@ -181,3 +197,54 @@ mod tests {
assert!(!echo_matches(&echo_request(7),7)); assert!(!echo_matches(&[255],7));
}
}
/// CaptureSettingRequest field 7 = ChromaFormat 3, wrapped in RPC 21/case 2.
pub fn request_chroma444()->Vec<u8>{
let mut header=Vec::new();int(1,2_000_000,&mut header);
let mut setting=Vec::new();int(7,3,&mut setting);
let mut rpc=Vec::new();bytes(1,&header,&mut rpc);bytes(2,&setting,&mut rpc);
envelope(2_000_000,21,&rpc)
}
/// Message 18 is an encoder capability LIST, not the chosen codec; RTP's negotiated PT is authoritative.
/// Message 25 reports actual encoded chroma, which may fall back to 4:2:0.
pub fn media_control(data:&[u8])->(Option<u32>,Option<u32>){
let mut codecs=None;let mut chroma=None;
if let Ok(fields)=decode_wire(data){for field in fields {
if let WireValue::Bytes(body)=field.value {match field.number {
18=>{if let Ok(caps)=decode_wire(body){let mut mask=0;for cap in caps {if let (1,WireValue::Bytes(cap))=(cap.number,cap.value){
match super::tools::uint(cap,3){1=>mask|=1,2=>mask|=2,_=>{}}
}}codecs=Some(mask);}},
25=>{let value=super::tools::uint(body,3);if [1,3].contains(&value){chroma=Some(value as u32);}},
_=>{},
}}
}}(codecs,chroma)
}
#[cfg(test)] mod video_tests {
use super::*;
#[test] fn decoder_caps_cover_both_codecs_and_chroma_and_force_h264(){
let original:Shape=serde_json::from_str(include_str!("../../tests/fixtures/viewer/session-shape.json")).unwrap();
for force in [false,true] {
let s=original.video_preferences(force);s.validate_session().unwrap();assert_eq!(s.chroma,1);
let options=s.connect_options("synthetic-device").unwrap();
let mut caps=Vec::new();
for f in decode_wire(&options).unwrap(){if let (4,WireValue::Bytes(cap))=(f.number,f.value){
assert_eq!(super::super::tools::uint(cap,1),30);
assert_eq!(super::super::tools::uint(cap,3),1280);assert_eq!(super::super::tools::uint(cap,4),720);
caps.push((super::super::tools::uint(cap,2),super::super::tools::uint(cap,5)));
}}
assert_eq!(caps,if force{vec![(1,1),(1,3)]}else{vec![(2,1),(2,3),(1,1),(1,3)]});
assert!(s.video_preferences(force).connect_options("synthetic-device").unwrap()==options);
}
}
#[test] fn chroma_request_and_actual_sync_are_independent_of_encoder_list(){
let request=request_chroma444();let rpc=super::super::tools::blob(&request,21).unwrap();
let setting=super::super::tools::blob(rpc,2).unwrap();assert_eq!(super::super::tools::uint(setting,7),3);
assert_eq!(decode_wire(setting).unwrap().len(),1);
let mut caps=Vec::new();for codec in [1,2] {let mut cap=Vec::new();int(3,codec,&mut cap);bytes(1,&cap,&mut caps);}
let mut msg=envelope(1,18,&caps);let mut sync=Vec::new();int(3,1,&mut sync);bytes(25,&sync,&mut msg);
assert_eq!(media_control(&msg),(Some(3),Some(1)));
assert_eq!(media_control(&envelope(2,25,&[24,3])),(None,Some(3)));
assert_eq!(media_control(&[255]),(None,None));
}
}
+9 -1
View File
@@ -4,6 +4,7 @@
#[path = "config.rs"] mod config;
#[path = "protocol.rs"] mod protocol;
mod viewer;
#[cfg(test)] #[path="hevc.rs"] mod hevc;
use viewer::clipboard;
#[path="clipboard_protocol.rs"] mod clipboard_protocol;
#[path="clipboard_rich.rs"] mod clipboard_rich;
@@ -12,7 +13,7 @@ use viewer::clipboard;
#[cfg(test)] #[path="initialization.rs"] mod initialization;
use viewer::{api::Api,rtc::ReadyPolicy,session::{Options,Session},wire::Shape};
use std::time::Duration;
const HELP:&str="Experimental native viewer.\n uuremote-viewer ui (GTK window: sign-in, devices, sessions) Desktop UI requires viewer-sdl (decodes with the ffmpeg CLI, or FFmpeg 6.x libraries with viewer-ffmpeg).\n uuremote-viewer power DEVICE reboot|shutdown\n uuremote-viewer terminal DEVICE [SESSION]\n uuremote-viewer files DEVICE ls PATH | get REMOTE LOCAL | put LOCAL REMOTE\n uuremote-viewer portmap DEVICE LOCAL_PORT HOST PORT\n uuremote-viewer files-ui|portmap-ui|terminal-ui DEVICE NAME (GTK tool windows)\n uuremote-viewer login [--no-gui]\n uuremote-viewer devices METHOD ARRAY_POINTER ID_POINTER NAME_POINTER\n uuremote-viewer assist CONNECT_ID (验证码从 stdin 读取一行)\n uuremote-viewer connect DEVICE true|false SHAPE_JSON echo|manual raw|legacy [--view-only]\nAudio: UU_VIEWER_AUDIO=0 disables playback. UU_VIEWER_VIEW_ONLY=1 disables desktop input.\nDevice-list method/JSON pointers and compatibility shape require verified operator configuration.\nQR login reuses the sibling uuremote-server login command. Never run login in a captured log.\nEcho readiness is transport observation only; no sendVideoTrack RPC is emitted.";
const HELP:&str="Experimental native viewer.\n uuremote-viewer ui (GTK window: sign-in, devices, sessions) Desktop UI requires viewer-sdl (decodes with the ffmpeg CLI, or FFmpeg 6.x libraries with viewer-ffmpeg).\n uuremote-viewer power DEVICE reboot|shutdown\n uuremote-viewer terminal DEVICE [SESSION]\n uuremote-viewer files DEVICE ls PATH | get REMOTE LOCAL | put LOCAL REMOTE\n uuremote-viewer portmap DEVICE LOCAL_PORT HOST PORT\n uuremote-viewer files-ui|portmap-ui|terminal-ui DEVICE NAME (GTK tool windows)\n uuremote-viewer login [--no-gui]\n uuremote-viewer devices METHOD ARRAY_POINTER ID_POINTER NAME_POINTER\n uuremote-viewer assist CONNECT_ID (验证码从 stdin 读取一行)\n uuremote-viewer connect DEVICE true|false SHAPE_JSON echo|manual raw|legacy [--view-only]\nVideo: UU_VIEWER_CODEC=h264 forces H.264; UU_VIEWER_CHROMA=444 requests true color (default 4:2:0).\nAudio: UU_VIEWER_AUDIO=0 disables playback. UU_VIEWER_VIEW_ONLY=1 disables desktop input.\nDevice-list method/JSON pointers and compatibility shape require verified operator configuration.\nQR login reuses the sibling uuremote-server login command. Never run login in a captured log.\nEcho readiness is transport observation only; no sendVideoTrack RPC is emitted.";
const DEFAULT_SHAPE:&str=r#"{"capture_type":1,"platform":3,"client_version":"4.42.0","width":1920,"height":1080,"fps":60,"chroma":1,
"decoders":[{"fps":60,"codec":1,"width":3840,"height":2160,"chroma":1}],"display_modes":[[1920,1080,60]],
"displays":[{"id":1,"fps":60,"type":0,"hdr":0}],
@@ -88,10 +89,17 @@ async fn desktop_command(args:&[String],code:Option<String>)->Result<(),String>
}else{None};
let creds=credentials()?;
let api=Api::new(auth::UUAuthClient::new(creds.clone()),auth::BASE_URL);
#[cfg(feature="viewer-sdl")]
let account_connection=code.is_none();
let room=match code {
Some(code)=>api.join_share(&args[1],&code).await,
None=>api.join(&args[1],force,true).await,
}.map_err(|e|e.to_string())?;
#[cfg(feature="viewer-sdl")]
if let Some((window,_))=desktop.as_mut().filter(|_|!view_only){
let device=account_connection.then_some(args[1].as_str());
window.set_mac(api.target_is_mac(&room,device).await.map_err(|e|e.to_string())?);
}
let options=Options{timeout:Duration::from_secs(12),legacy_prefix:legacy,ready_policy:policy};
let mut session=Session::open(&room,&shape,&creds.device_id,&creds.client_id,options,Default::default()).await?;
#[cfg(feature="viewer-sdl")]
+15
View File
@@ -0,0 +1,15 @@
The `red-{420,444}.{h264,hevc}` fixtures contain six synthetic 32x32 red IDRs,
not desktop captures. Their decoded pixel formats are yuv420p and yuv444p.
They let the CLI decoder test run without requiring encoders at test time.
Generated with FFmpeg (substitute `420` / `444` for `CHROMA`):
```sh
ffmpeg -f lavfi -i color=c=red:s=32x32:r=10 -frames:v 6 -pix_fmt yuvCHROMAp \
-c:v libx264 -threads 1 -preset ultrafast -tune zerolatency -g 1 \
-f h264 red-CHROMA.h264
ffmpeg -f lavfi -i color=c=red:s=32x32:r=10 -frames:v 6 -pix_fmt yuvCHROMAp \
-c:v libx265 -threads 1 -preset ultrafast -tune zerolatency -g 1 \
-x265-params pools=none:frame-threads=1:log-level=none:info=0 \
-f hevc red-CHROMA.hevc
```
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.