Compare commits

..
7 Commits
Author SHA1 Message Date
VallierDevandClaude Opus 5.5 1e94175414 Release v0.2.33
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 19:02:33 +08:00
VallierDevandClaude Opus 5.5 cfe9601874 WebRTC: restart a promoted desktop's video after its stream ended (#114)
A tool peer promoted by capture_desktop stayed marked as the desktop after its
encoder exited on its own (Wayland capture failing while locked/DPMS off), so a
later capture_desktop on the same peer only logged and never restarted the
video. Treat a finished stream task as ended: capture_desktop asks for the
desktop again and promote() restarts the video on the existing input lease.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 18:02:56 +08:00
VallierDevandClaude Opus 5.5 ab5d0a656b Release v0.2.32
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 22:54:59 +08:00
VallierDevandClaude Opus 5.5 dcc67ed550 WebRTC: recompute the peer state from the real DTLS transport after ICE recovers
The ICE transport state callback was created before the DTLS transport and
captured the default placeholder, whose state never leaves New. Every later
ICE change therefore computed the peer state against a "New" DTLS: after a
brief ICE Disconnected -> Connected the PeerConnection settled on New instead
of Connected, so the idle room renewal treated the still-working
port-mapping peer as idle and tore the room down.

Wire the callback to the real DTLS transport through a weak slot filled right
after it is created.

Fixes #109

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 22:53:22 +08:00
liuyanghejerry bf7bd50062 viewer: forward mouse wheel events as mouse_scroll (#108)
SDL MouseWheel events had no arm in the event translator, so wheel input was dropped. Send one notch as 120 wire units (the Mac convention), use SDL's precise deltas for trackpads, honour the flipped direction for natural scrolling, and keep the existing focus and view-only gating.
2026-10-24 22:42:53 +08:00
liuyanghejerry 287b21a622 viewer: map pointer input to the content rectangle of the padded canvas (#107)
The CLI decoder pads every frame into a fixed 1920x1080 canvas so rawvideo frames have a known byte length. Input mapping and the SDL viewport treated that padded canvas as the picture, so non-16:9 hosts (official Macs are usually 16:10) had the pillarbox bars inside the 0..1 range: the remote pointer lagged inward and the error grew towards the screen edges.

Parse the coded picture size from ffmpeg's one-time "Video:" stream line, carry it on Frame.content, and render/map only the content sub-rectangle. 16:9 streams are unchanged; UU_VIEWER_SIZE keeps working.
2026-10-24 21:41:48 +08:00
Mr. ErrorandClaude Code bcde9f967c Signaling: keep heartbeat detection alive when delivery stalls (#104)
* Signaling: keep heartbeat detection alive when delivery stalls

The receive loop awaited event_sender.send() without a bound. When the main loop was parked in PeerConnection::close after a released peer, the 64-slot event queue filled and the receiver blocked inside the loop body, so the Engine.IO deadline in the same select! never ran. The host stayed online with a half-closed socket for hours.

- add a watchdog over the last inbound frame, independent of the receive loop
- bound event, pong and control sends with a timeout or try_send
- timeout PeerConnection::close so one dead peer cannot stall signaling
- regression test with a blocked event consumer

Co-Authored-By: Claude Code <[email protected]>

* Signaling: always report Disconnected when the receiver stops

The receive loop can break without enqueueing Disconnected: event delivery exceeded SEND_STALL_TIMEOUT, or the writer already stopped. receiver_done then lets the watchdog exit, while the main loop's rx_event stays open because the idle-renewal timer holds a sender. The host stays in a dead room until the 15-minute renewal, which is deferred while a peer is connected.

Hand a Disconnected off from a detached task after every loop exit. Duplicates are harmless: the main loop breaks on the first one and drops the old receiver.

The new test keeps the consumer blocked past SEND_STALL_TIMEOUT, then drains the queue and expects Disconnected.

Co-Authored-By: Claude Code <[email protected]>

---------

Co-authored-by: Claude Code <[email protected]>
2026-10-24 16:38:43 +08:00
14 changed files with 381 additions and 76 deletions

No files matched your search

Generated
+1 -1
View File
@@ -2741,7 +2741,7 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uu-remote-linux"
version = "0.2.31"
version = "0.2.33"
dependencies = [
"async-trait",
"base64 0.22.1",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "uu-remote-linux"
version = "0.2.31"
version = "0.2.33"
edition = "2021"
default-run = "uuremote-server"
authors = ["uu-remote-linux contributors"]
+17
View File
@@ -0,0 +1,17 @@
修复端口映射在 ICE 瞬断恢复后被续房拆掉、信令卡死后长时间假在线,以及查看器 CLI 的指针偏移与滚轮。/ Fixes port-mapping teardown after ICE recovery, stalled signaling, and viewer CLI pointer/wheel input.
### 修复 / Fixed
- **ICE 瞬断恢复后连接被拆(#109,感谢 @sinoyster 的细致分析)**:WebRTC 的 ICE 状态回调读到的是占位 DTLS 对象,ICE `Disconnected → Connected` 恢复后 PeerConnection 停在 `New`,15 分钟续房把仍在使用的端口映射当成空闲,拆房并断开所有映射 TCP 流。现在恢复后会回到 `Connected`。
After a brief ICE drop the peer state settled on `New` instead of `Connected`, so the idle room renewal tore down live port-mapping streams.
- **信令投递卡住时心跳检测失效(#104,感谢 @RiverOnVenus)**:事件队列阻塞时 Engine.IO 超时不再执行,主机可能带着半关闭的连接“在线”数小时。现在有独立看门狗、发送超时和关闭超时。
Heartbeat detection keeps running when event delivery stalls; the host reconnects instead of staying half-online.
- **查看器 CLI**:16:10 等非 16:9 主机的指针按内容区域映射,不再内偏(#107,感谢 @liuyanghejerry);鼠标滚轮/触控板转发为滚动(#108,感谢 @liuyanghejerry)。
Viewer CLI: pointer mapped to the content rectangle of the padded canvas; mouse wheel forwarded.
### 验证范围 / Validation
单元测试与 CI;#109 的 ICE 恢复场景尚未实机复现,欢迎反馈。
### 下载 / Download
`uu-remote-linux_0.2.32_<amd64|arm64>.deb`,或 `uu-remote-linux-v0.2.32-<x86_64|aarch64>-linux.tar.gz`(解压后运行 `./install.sh`),或 `docker pull ghcr.io/vallierdev/uu-remote-linux:v0.2.32`(x86_64)。
**非官方项目,与网易无关。** / **Unofficial, not affiliated with NetEase.**
+13
View File
@@ -0,0 +1,13 @@
修复 Wayland 锁屏或熄屏后,由工具连接(端口映射)升级的桌面视频结束后无法再次打开。/ Fixes a promoted tool connection whose desktop video could not be reopened after the screen locked or blanked.
### 修复 / Fixed
- **锁屏/熄屏后桌面无法重新打开(#114,感谢 @sinoyster 的完整报告)**:通过 capture_desktop 升级为桌面的工具连接上,若 Wayland 采集因锁屏或 DPMS 关闭而结束,再次 capture_desktop 不会重建视频,查看器只剩黑屏。现在会重新请求桌面并在原连接上重启视频,保留原输入租约。
On a tool connection promoted with capture_desktop, a desktop stream that ended (e.g. Wayland capture failing on lock/DPMS) is now restarted on the next capture_desktop, keeping the original input lease.
### 验证范围 / Validation
单元测试与 CI;#114 的锁屏场景尚未实机复现,欢迎反馈。
### 下载 / Download
`uu-remote-linux_0.2.33_<amd64|arm64>.deb`,或 `uu-remote-linux-v0.2.33-<x86_64|aarch64>-linux.tar.gz`(解压后运行 `./install.sh`),或 `docker pull ghcr.io/vallierdev/uu-remote-linux:v0.2.33`(x86_64)。
**非官方项目,与网易无关。** / **Unofficial, not affiliated with NetEase.**
+9
View File
@@ -244,6 +244,15 @@ async fn capture_desktop_on_a_port_mapping_peer_promotes_it_once() {
let lease = host.role.input().unwrap();
host.promote(None);
assert!(lease.is_current());
// Its video ended on its own (capture failed while locked, #114): capture_desktop restarts it
// on the same input lease.
let ended = tokio::spawn(async {});
*host.role.stream_task.lock().unwrap() = Some(ended.abort_handle());
ended.await.unwrap();
send(&viewer, "CONTROL_DATA_CHANNEL", crate::initialization::envelope(9, 0, 3, &[8, 4])).await;
tokio::time::timeout(Duration::from_secs(5), requested.recv()).await.unwrap().unwrap();
host.promote(None);
assert!(host.role.is_desktop() && !host.role.stream_ended() && lease.is_current());
// A newer desktop demotes it; the port mapping stays, and capture_desktop promotes it again.
host.demote();
assert!(!host.role.is_desktop() && host.role.input().is_none());
+15 -7
View File
@@ -73,6 +73,14 @@ fn superseded_peer_ids(sessions: &HashMap<String, WebRtcSession>, ice: &str, des
sessions.iter().filter(|(id, peer)| id.as_str() == ice || (desktop && peer.desktop))
.map(|(id, _)| id.clone()).collect()
}
/// A peer whose transport is already broken can leave `PeerConnection::close`
/// waiting indefinitely; signaling must keep flowing without it.
async fn close_peer(pc: &::webrtc::peer_connection::RTCPeerConnection) {
if tokio::time::timeout(std::time::Duration::from_secs(5), pc.close()).await.is_err() {
log::warn!("Peer connection close timed out; continuing");
}
}
fn released_peer_ids<T>(sessions: &HashMap<String, T>, ice: Option<&serde_json::Value>) -> Vec<String> {
let ice = ice.map(|id| id.as_str().map(str::to_owned).unwrap_or_else(|| id.to_string()));
sessions.keys().filter(|id| ice.as_deref().map_or(true, |ice| id.as_str() == ice)).cloned().collect()
@@ -685,7 +693,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
if let Some(task) = command_task { task.abort(); }
// One exit path for commands, signals, transport failures and UI failures.
if let Some(session) = assist.as_mut() { session.allowed = false; }
for (_, old) in webrtc_sessions.lock().await.drain() { let _ = old.pc.close().await; }
for (_, old) in webrtc_sessions.lock().await.drain() { close_peer(&old.pc).await; }
virtual_audio::disable(); input.shutdown(); terminal::shutdown_all();
// Switching identity (signed in / out) restarts at once and the new identity republishes the same
// device share: logout already withdrew it, and after a QR login the cloud moved it to the account
@@ -858,7 +866,7 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
::webrtc::peer_connection::peer_connection_state::RTCPeerConnectionState::Closed |
::webrtc::peer_connection::peer_connection_state::RTCPeerConnectionState::Failed)).map(|(id, _)| id.clone()).collect();
let mut desktop_ended = false;
for id in ended { if let Some(s) = sessions.remove(&id) { desktop_ended |= s.role.is_desktop(); let _ = s.pc.close().await; } }
for id in ended { if let Some(s) = sessions.remove(&id) { desktop_ended |= s.role.is_desktop(); close_peer(&s.pc).await; } }
if desktop_ended && !sessions.values().any(|s| s.role.is_desktop()) { if let Some(s) = assist.as_mut() { s.session_ended()?; } }
if let Some(s) = assist.as_mut() { s.tick()?; }
}
@@ -946,7 +954,7 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
::webrtc::peer_connection::peer_connection_state::RTCPeerConnectionState::Closed |
::webrtc::peer_connection::peer_connection_state::RTCPeerConnectionState::Failed)).map(|(k, _)| k.clone()).collect();
let mut desktop_ended = false;
for k in ended { if let Some(old) = sessions.remove(&k) { desktop_ended |= old.role.is_desktop(); let _ = old.pc.close().await; } }
for k in ended { if let Some(old) = sessions.remove(&k) { desktop_ended |= old.role.is_desktop(); close_peer(&old.pc).await; } }
if desktop_ended && !sessions.values().any(|s| s.role.is_desktop()) { if let Some(s) = assist.as_mut() { s.session_ended()?; } }
if let Some(s) = assist.as_mut() { s.tick()?; }
@@ -961,7 +969,7 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
if desktop { video::warm_auto_probe(); video::check_ffmpeg_usable(); }
// Replace only this ICE id or a previous desktop, never tool peers.
for id in superseded_peer_ids(&sessions, &ice, desktop) {
if let Some(old) = sessions.remove(&id) { let _ = old.pc.close().await; }
if let Some(old) = sessions.remove(&id) { close_peer(&old.pc).await; }
}
// A tool peer promoted by capture_desktop stops its desktop but keeps its tools.
if desktop { for old in sessions.values() { old.demote(); } }
@@ -1014,7 +1022,7 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
}
Err(_) => {
log::error!("Failed to handle offer (details omitted)");
if let Some(session) = sessions.remove(&ice) { let _ = session.pc.close().await; }
if let Some(session) = sessions.remove(&ice) { close_peer(&session.pc).await; }
}
}},
Err(_) => log::error!("Create peer failed (details omitted)"),
@@ -1037,7 +1045,7 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
for k in targets {
if let Some(old) = sessions.remove(&k) {
desktop_ended |= old.role.is_desktop();
if let Err(e) = old.pc.close().await { log::warn!("Close released peer failed: {}", e); }
close_peer(&old.pc).await;
}
}
if desktop_ended && !sessions.values().any(|s| s.role.is_desktop()) { if let Some(s) = assist.as_mut() { s.session_ended()?; } }
@@ -1075,7 +1083,7 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
let superseded: Vec<String> = sessions.iter().filter(|(id, old)| id.as_str() != ice && old.role.is_desktop())
.map(|(id, _)| id.clone()).collect();
for id in superseded {
if sessions[&id].desktop { if let Some(old) = sessions.remove(&id) { let _ = old.pc.close().await; } }
if sessions[&id].desktop { if let Some(old) = sessions.remove(&id) { close_peer(&old.pc).await; } }
else { sessions[&id].demote(); }
}
if let Some(peer) = sessions.get(&ice) { peer.promote(screen); }
+164 -37
View File
@@ -3,12 +3,20 @@
use futures_util::{SinkExt, StreamExt};
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::mpsc;
use tokio_tungstenite::connect_async;
use tokio_tungstenite::tungstenite::client::IntoClientRequest;
use tokio_tungstenite::tungstenite::Message;
/// A peer that stops reading must not park the heartbeat loop on a full queue.
/// Kept below the Engine.IO window so the protocol deadline still fires.
const SEND_STALL_TIMEOUT: Duration = Duration::from_secs(10);
/// How often the watchdog checks the last inbound frame.
const WATCHDOG_TICK: Duration = Duration::from_secs(1);
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SoacMessage {
#[serde(rename = "type")]
@@ -70,6 +78,11 @@ fn refreshed_key(ack:&Value)->Option<String> {
(!key.is_empty() && key.len()<=16384).then(||key.to_owned())
}
/// Deliver an event without letting a stalled consumer hide a dead socket.
async fn send_event(sender: &mpsc::Sender<SignalingEvent>, event: SignalingEvent) -> bool {
matches!(tokio::time::timeout(SEND_STALL_TIMEOUT, sender.send(event)).await, Ok(Ok(())))
}
#[cfg(test)]
mod refresh_tests {
use super::refreshed_key;
@@ -128,6 +141,73 @@ mod refresh_tests {
assert!(disconnected.is_ok(),"a socket without pings must be reported dead");
drop(hold_tx);
}
#[tokio::test]
async fn blocked_event_consumer_does_not_hide_a_dead_socket() {
use futures_util::{SinkExt,StreamExt};
use tokio_tungstenite::tungstenite::Message;
let listener=tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address=listener.local_addr().unwrap();
let (closed_tx,closed_rx)=tokio::sync::oneshot::channel::<()>();
tokio::spawn(async move {
let (tcp,_)=listener.accept().await.unwrap();
let socket=tokio_tungstenite::accept_async(tcp).await.unwrap();
let (mut sink,mut stream)=socket.split();
// Shortest accepted window (clamped to 5 s); never ping, only events.
sink.send(Message::Text(r#"0{"pingInterval":1000,"pingTimeout":1000}"#.into())).await.unwrap();
let _=stream.next().await;
sink.send(Message::Text("40{}".into())).await.unwrap();
loop {
tokio::select! {
r=stream.next()=>match r {
None|Some(Err(_))|Some(Ok(Message::Close(_)))=>{let _=closed_tx.send(());return;}
_=>{}
},
_=tokio::time::sleep(std::time::Duration::from_millis(50))=>{
if sink.send(Message::Text(r#"42["bmsg_push",{}]"#.into())).await.is_err() {
let _=closed_tx.send(());return;
}
}
}
}
});
// Capacity 1 and no reader: the first event fills the queue, so the
// receive loop would park inside `send().await` without the watchdog.
let (sender,_events)=tokio::sync::mpsc::channel(1);
let _client=super::SignalingClient::connect(&format!("ws://{address}"),"synthetic-only",sender).await.unwrap();
let closed=tokio::time::timeout(std::time::Duration::from_secs(15),closed_rx).await;
assert!(closed.is_ok(),"a blocked consumer must not keep a dead signaling socket open");
}
#[tokio::test]
async fn stalled_event_delivery_still_reports_disconnected() {
use futures_util::{SinkExt,StreamExt};
use tokio_tungstenite::tungstenite::Message;
let listener=tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address=listener.local_addr().unwrap();
tokio::spawn(async move {
let (tcp,_)=listener.accept().await.unwrap();
let socket=tokio_tungstenite::accept_async(tcp).await.unwrap();
let (mut sink,mut stream)=socket.split();
// A long window: the send stall must fire before the heartbeat deadline.
sink.send(Message::Text(r#"0{"pingInterval":30000,"pingTimeout":30000}"#.into())).await.unwrap();
let _=stream.next().await;
sink.send(Message::Text("40{}".into())).await.unwrap();
loop {
tokio::select! {
r=stream.next()=>{ if matches!(r,None|Some(Err(_))|Some(Ok(Message::Close(_)))){return} }
_=tokio::time::sleep(std::time::Duration::from_millis(50))=>{
if sink.send(Message::Text(r#"42["bmsg_push",{}]"#.into())).await.is_err(){return}
}
}
}
});
let (sender,mut events)=tokio::sync::mpsc::channel(1);
let _client=super::SignalingClient::connect(&format!("ws://{address}"),"synthetic-only",sender).await.unwrap();
// Let SEND_STALL_TIMEOUT expire while nothing drains the queue.
tokio::time::sleep(std::time::Duration::from_secs(12)).await;
assert!(matches!(events.recv().await,Some(super::SignalingEvent::Connected)));
let disconnected=tokio::time::timeout(std::time::Duration::from_secs(5),events.recv()).await;
assert!(matches!(disconnected,Ok(Some(super::SignalingEvent::Disconnected))),"a stalled delivery must still report Disconnected");
}
}
impl SignalingClient {
@@ -158,17 +238,25 @@ impl SignalingClient {
log::info!("Connected to Engine.IO / Socket.IO");
let (mut write, mut read) = ws_stream.split();
let (tx_outgoing, mut rx_outgoing) = mpsc::channel::<String>(32);
let started = Instant::now();
let last_rx = Arc::new(AtomicU64::new(0));
let ping_window_ms = Arc::new(AtomicU64::new(45_000));
let receiver_done = Arc::new(AtomicBool::new(false));
// 发送协程
let writer=tokio::spawn(async move {
while let Some(msg) = rx_outgoing.recv().await {
if let Err(e) = write.send(Message::Text(msg)).await {
let _ = e;
log::warn!("WS send failed (details omitted)");
break;
match tokio::time::timeout(SEND_STALL_TIMEOUT, write.send(Message::Text(msg))).await {
Ok(Ok(())) => {}
Ok(Err(_)) | Err(_) => {
// A peer that stopped reading must not park every producer.
log::warn!("WS send failed or stalled (details omitted)");
break;
}
}
}
});
let writer_abort = writer.abort_handle();
let tx_out_clone = tx_outgoing.clone();
static GENERATION: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1);
@@ -177,9 +265,12 @@ impl SignalingClient {
let refreshed=reconnect_key.clone();
// 接收与状态机协程
tokio::spawn(async move {
let watchdog_events = event_sender.clone();
let watchdog_last_rx = last_rx.clone();
let watchdog_ping_window_ms = ping_window_ms.clone();
let watchdog_done = receiver_done.clone();
let receiver = tokio::spawn(async move {
let mut req_counter = 10;
let started=std::time::Instant::now();
let mut last_ping=started;
let mut pings=0u64;
let mut pong_queued=0u64;
@@ -202,7 +293,7 @@ impl SignalingClient {
_=event_sender.closed()=>break,
_=tokio::time::sleep_until(ping_deadline) => {
log::warn!("Signaling heartbeat lost: generation={} age_s={} pings={} last_ping_gap_s={}",generation,started.elapsed().as_secs(),pings,last_ping.elapsed().as_secs());
let _ = event_sender.send(SignalingEvent::Disconnected).await;
let _ = send_event(&event_sender, SignalingEvent::Disconnected).await;
break;
},
// Official ControlledSession RefreshSignalTokenLoop: first
@@ -213,8 +304,8 @@ impl SignalingClient {
refresh_acks.push_back(req_counter);
if refresh_acks.len()>16 { refresh_acks.pop_front(); }
let packet=format!("42{}[\"refresh_reconnect_key\"]",req_counter);
if tx_out_clone.send(packet).await.is_err() {
let _=event_sender.send(SignalingEvent::Disconnected).await;break;
if tx_out_clone.try_send(packet).is_err() {
let _=send_event(&event_sender, SignalingEvent::Disconnected).await;break;
}
next_refresh=tokio::time::Instant::now()+std::time::Duration::from_secs(30);
log::info!("Signaling reconnect-key refresh requested: generation={} age_s={}",generation,started.elapsed().as_secs());
@@ -222,9 +313,14 @@ impl SignalingClient {
},
msg=read.next()=>match msg {
Some(msg)=>msg,
None=>{let _=event_sender.send(SignalingEvent::Disconnected).await;break;}
None=>{let _=send_event(&event_sender, SignalingEvent::Disconnected).await;break;}
},
};
// Any inbound frame proves the socket is alive. A reader parked on
// event delivery stops updating this, which the watchdog catches.
if msg_res.is_ok() {
last_rx.store(started.elapsed().as_millis() as u64, Ordering::Relaxed);
}
match msg_res {
Ok(Message::Text(txt)) => {
// Engine.IO 报文处理
@@ -233,16 +329,20 @@ impl SignalingClient {
log::info!("Signaling heartbeat config: generation={} ping_interval_ms={:?} ping_timeout_ms={:?}",generation,open.get("pingInterval").and_then(Value::as_u64),open.get("pingTimeout").and_then(Value::as_u64));
if let (Some(interval),Some(timeout))=(open.get("pingInterval").and_then(Value::as_u64),open.get("pingTimeout").and_then(Value::as_u64)) {
ping_window=std::time::Duration::from_millis(interval.saturating_add(timeout).clamp(5_000,300_000));
ping_window_ms.store(ping_window.as_millis() as u64, Ordering::Relaxed);
ping_deadline=tokio::time::Instant::now()+ping_window;
}
}
// Open packet -> 发送 Socket.IO connect
let _ = tx_out_clone.send("40".into()).await;
if tx_out_clone.try_send("40".into()).is_err() {
let _=send_event(&event_sender, SignalingEvent::Disconnected).await;break;
}
} else if txt == "2" {
// Ping -> 回复 Pong
let gap=last_ping.elapsed().as_secs();last_ping=std::time::Instant::now();pings+=1;
ping_deadline=tokio::time::Instant::now()+ping_window;
if tx_out_clone.send("3".into()).await.is_ok() { pong_queued+=1; }
if tx_out_clone.try_send("3".into()).is_ok() { pong_queued+=1; }
else { let _=send_event(&event_sender, SignalingEvent::Disconnected).await;break; }
if pings==1 || pings%12==0 {
log::info!("Signaling heartbeat progress: generation={} age_s={} pings={} pong_queued={} gap_s={}",generation,started.elapsed().as_secs(),pings,pong_queued,gap);
}
@@ -252,8 +352,10 @@ impl SignalingClient {
// Socket.IO connected -> 发送 room_info 请求
req_counter += 1;
let room_req = format!("42{}[\"room_info\"]", req_counter);
let _ = tx_out_clone.send(room_req).await;
let _ = event_sender.send(SignalingEvent::Connected).await;
if tx_out_clone.try_send(room_req).is_err() {
let _=send_event(&event_sender, SignalingEvent::Disconnected).await;break;
}
if !send_event(&event_sender, SignalingEvent::Connected).await { break; }
} else if txt.starts_with("43") {
// Socket.IO ack 回调
let begin=txt.find('[').unwrap_or(2);
@@ -272,7 +374,7 @@ impl SignalingClient {
next_refresh=tokio::time::Instant::now()+std::time::Duration::from_secs(300);
}
log::info!("Signaling reconnect-key refresh acknowledged: generation={} age_s={} success={}",generation,started.elapsed().as_secs(),success);
} else { let _ = event_sender.send(SignalingEvent::RoomInfoAck(v)).await; }
} else if !send_event(&event_sender, SignalingEvent::RoomInfoAck(v)).await { break; }
}
} else if txt.starts_with("45") || txt.starts_with("46") {
// Socket.IO binary event/ack: header text, attachments follow as binary frames.
@@ -296,14 +398,10 @@ impl SignalingClient {
match event_name {
"bmsg_push" => {
let _ = event_sender
.send(SignalingEvent::DeviceInfoChanged(data.clone()))
.await;
if !send_event(&event_sender, SignalingEvent::DeviceInfoChanged(data.clone())).await { break; }
}
"be-controlled" => {
let _ = event_sender
.send(SignalingEvent::BeControlled(data.clone()))
.await;
if !send_event(&event_sender, SignalingEvent::BeControlled(data.clone())).await { break; }
}
"soac" => {
// Offers only: structure (keys, types, lengths), never SDP text or ids.
@@ -326,9 +424,7 @@ impl SignalingClient {
if soac.client_id.is_none() {
soac.client_id = client_id_opt;
}
let _ = event_sender
.send(SignalingEvent::Soac(soac))
.await;
if !send_event(&event_sender, SignalingEvent::Soac(soac)).await { break; }
}
Err(e) => {
let _ = e;
@@ -338,20 +434,16 @@ impl SignalingClient {
}
"released" => {
log::info!("Cloud peer event: generation={} event=released age_s={} pings={} last_ping_gap_s={} ice_id_present={}",generation,started.elapsed().as_secs(),pings,last_ping.elapsed().as_secs(),data.get("ice_id").is_some());
let _ = event_sender
.send(SignalingEvent::Released(data.clone()))
.await;
if !send_event(&event_sender, SignalingEvent::Released(data.clone())).await { break; }
}
"left" | "leave" => {
log::info!("Cloud peer event: generation={} event={} age_s={} pings={} last_ping_gap_s={} ice_id_present={}",generation,event_name,started.elapsed().as_secs(),pings,last_ping.elapsed().as_secs(),data.get("ice_id").is_some());
let ice_id = data.get("ice_id").and_then(|v| v.as_str()).unwrap_or("");
if ice_id.is_empty() {
log::warn!("Cloud room leave without peer identifier; renewing signaling (expiry cause unknown)");
let _ = event_sender.send(SignalingEvent::Disconnected).await;
if !send_event(&event_sender, SignalingEvent::Disconnected).await { break; }
} else {
let _ = event_sender
.send(SignalingEvent::Left(data.clone()))
.await;
if !send_event(&event_sender, SignalingEvent::Left(data.clone())).await { break; }
}
}
_ => {
@@ -401,28 +493,60 @@ impl SignalingClient {
let controller_mac = platform.is_some_and(|v| v.as_i64() == Some(4)
|| v.as_str().is_some_and(|s| s == "4" || s.to_ascii_lowercase().contains("mac") || s.to_ascii_lowercase().contains("darwin")));
log::info!("Controller platform: {} mac={}", platform.map(|v| v.to_string()).unwrap_or_default().chars().take(24).collect::<String>(), controller_mac);
let _ = event_sender.send(SignalingEvent::ConnectRequest { capture_type, ice_id, transport:crate::session_ice::SessionIce::from_header(&header), controller_mac }).await;
if !send_event(&event_sender, SignalingEvent::ConnectRequest { capture_type, ice_id, transport:crate::session_ice::SessionIce::from_header(&header), controller_mac }).await { break; }
}
}
}
Ok(Message::Close(frame)) => {
log::warn!("Signaling socket closed: generation={} age_s={} close_code={:?} pings={} last_ping_gap_s={}",generation,started.elapsed().as_secs(),frame.map(|f|u16::from(f.code)),pings,last_ping.elapsed().as_secs());
let _ = event_sender.send(SignalingEvent::Disconnected).await;
let _ = send_event(&event_sender, SignalingEvent::Disconnected).await;
break;
}
Err(_) => {
log::warn!("Signaling socket receive failed: generation={} age_s={} pings={} last_ping_gap_s={}",generation,started.elapsed().as_secs(),pings,last_ping.elapsed().as_secs());
let _ = event_sender.send(SignalingEvent::Disconnected).await;
let _ = send_event(&event_sender, SignalingEvent::Disconnected).await;
break;
}
_ => {}
}
}
// A stalled event queue or a writer that already stopped can end this
// loop without a delivered Disconnected. Hand one off from a detached
// task so the main loop always learns the room is dead; duplicates are
// harmless because it breaks on the first one.
let events = event_sender.clone();
tokio::spawn(async move { let _ = events.send(SignalingEvent::Disconnected).await; });
receiver_done.store(true, Ordering::Relaxed);
log::info!("Signaling receiver stopped: generation={} age_s={}",generation,started.elapsed().as_secs());
// Live peer callbacks can retain the old SignalingClient sender.
// Explicitly drop its write half too when this room stops reading.
writer.abort();
});
let receiver_abort = receiver.abort_handle();
// Independent of the receive loop: a consumer that stops draining events
// parks the loop inside an `.await`, where the ping deadline can no longer
// be observed. The watchdog closes the socket anyway.
tokio::spawn(async move {
let mut tick = tokio::time::interval(WATCHDOG_TICK);
tick.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
loop {
tick.tick().await;
if watchdog_done.load(Ordering::Relaxed) || watchdog_events.is_closed() { return; }
let window_ms = watchdog_ping_window_ms.load(Ordering::Relaxed);
let idle_ms = (started.elapsed().as_millis() as u64)
.saturating_sub(watchdog_last_rx.load(Ordering::Relaxed));
if idle_ms <= window_ms { continue; }
log::warn!("Signaling heartbeat lost (watchdog): idle_s={} window_ms={}", idle_ms/1000, window_ms);
// Deliver from a detached task: the receiver may be the one parked
// on a full queue, and the main loop still needs this event.
let events = watchdog_events.clone();
tokio::spawn(async move { let _ = events.send(SignalingEvent::Disconnected).await; });
receiver_abort.abort();
writer_abort.abort();
return;
}
});
Ok(Self { tx_outgoing, _reconnect_key:reconnect_key })
}
@@ -430,7 +554,10 @@ impl SignalingClient {
pub async fn emit_event(&self, event: &str, payload: serde_json::Value) -> Result<(), String> {
let packet = format!("42{}", serde_json::json!([event, payload]));
log::info!("Emitting signaling event: {}", event);
self.tx_outgoing.send(packet).await.map_err(|e| e.to_string())
tokio::time::timeout(SEND_STALL_TIMEOUT, self.tx_outgoing.send(packet))
.await
.map_err(|_| "signaling send stalled".to_string())?
.map_err(|e| e.to_string())
}
/// 发送 SOAC 消息(Answer 或 Candidate)
@@ -445,9 +572,9 @@ impl SignalingClient {
};
let msg = format!("42[\"soac\",{}]", payload);
log::info!("Emitting soac packet: type={}", soac.msg_type);
self.tx_outgoing
.send(msg)
tokio::time::timeout(SEND_STALL_TIMEOUT, self.tx_outgoing.send(msg))
.await
.map_err(|_| "Send soac timed out".to_string())?
.map_err(|e| format!("Send soac error: {}", e))
}
}
+1 -1
View File
@@ -37,7 +37,7 @@ impl Decoder for Ffmpeg {
let mut rgb=av::frame::Video::empty();scaler.run(&decoded,&mut rgb).map_err(|_|"Frame conversion failed")?;
let mut data=Vec::with_capacity(w as usize*h as usize*3);
for row in 0..h as usize {let start=row*rgb.stride(0);data.extend_from_slice(&rgb.data(0)[start..start+w as usize*3]);}
frames.push(Frame{width:w,height:h,rgb:data});
frames.push(Frame{width:w,height:h,rgb:data,content:None});
} Ok(frames)
}
}
+46 -8
View File
@@ -1,13 +1,16 @@
//! H.264/H.265 decoding through an external `ffmpeg` process, like the host's encoder pipeline.
//! Needs only the ffmpeg CLI, not FFmpeg development libraries.
//! Output is scaled into a fixed letterboxed size so every frame has a known byte length.
//! Output is scaled into a fixed letterboxed canvas so every frame has a known byte length; the
//! source size from ffmpeg's stream line tells the window which sub-rectangle is picture content.
use super::desktop::{Decoder,Frame};
use super::media::Codec;
use std::io::{Read,Write};
use std::io::{BufRead,Read,Write};
use std::process::{Child,ChildStdin,Command,Stdio};
use std::sync::Arc;
use std::sync::atomic::{AtomicU64,Ordering};
use std::sync::mpsc::{Receiver,TryRecvError};
pub struct Cli { child:Child, input:Option<ChildStdin>, codec:Codec, frames:Receiver<Vec<u8>>, width:u32, height:u32 }
pub struct Cli { child:Child, input:Option<ChildStdin>, codec:Codec, frames:Receiver<Vec<u8>>, width:u32, height:u32, source:Arc<AtomicU64> }
impl Cli {
/// `UU_VIEWER_SIZE=WxH` picks the decoded size (default 1920x1080); `UU_VIEWER_FFMPEG` the binary.
@@ -23,16 +26,27 @@ impl Cli {
let filter=format!("scale={width}:{height}:force_original_aspect_ratio=decrease,pad={width}:{height}:(ow-iw)/2:(oh-ih)/2");
let passthrough=passthrough_args(&ffmpeg);
let mut child=Command::new(&ffmpeg)
.args(["-hide_banner","-loglevel","error","-nostdin","-threads","1","-probesize","32","-analyzeduration","0",
.args(["-hide_banner","-loglevel","info","-nostats","-nostdin","-threads","1","-probesize","32","-analyzeduration","0",
"-fflags","nobuffer","-flags","low_delay","-f",codec.demuxer(),"-i","pipe:0","-an","-vf",&filter,
"-pix_fmt","rgb24",
// Raw H.264/H.265 carries no timestamps; constant-rate output dropped almost every frame of
// the official Mac host's stream (270 → 2). Pass frames through unchanged.
passthrough.0,passthrough.1,"-f","rawvideo","pipe:1"])
.stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::null())
.stdin(Stdio::piped()).stdout(Stdio::piped()).stderr(Stdio::piped())
.spawn().map_err(|e|format!("Start ffmpeg decoder: {e}"))?;
let input=child.stdin.take().ok_or("ffmpeg stdin missing")?;
let mut output=child.stdout.take().ok_or("ffmpeg stdout missing")?;
// Source picture size from the one-time stream line; 0 until parsed. Only the numbers are kept.
let source=Arc::new(AtomicU64::new(0));
if let Some(stderr)=child.stderr.take() {
let source=source.clone();
let _=std::thread::Builder::new().name("uu-viewer-ffmpeg-info".into()).spawn(move||{
for line in std::io::BufReader::new(stderr).lines().map_while(Result::ok) {
if source.load(Ordering::Relaxed)!=0 {continue}
if let Some((w,h))=parse_video_size(&line) {source.store((u64::from(w)<<32)|u64::from(h),Ordering::Relaxed);}
}
});
}
let size=width as usize*height as usize*3;
// Bounded: a slow window drops decoded frames instead of growing memory.
let (tx,frames)=std::sync::mpsc::sync_channel(2);
@@ -43,10 +57,23 @@ impl Cli {
match tx.try_send(frame) { Ok(())|Err(std::sync::mpsc::TrySendError::Full(_))=>{}, Err(_)=>break }
}
}).map_err(|e|format!("Decoder reader: {e}"))?;
Ok(Self{child,input:Some(input),codec,frames,width,height})
Ok(Self{child,input:Some(input),codec,frames,width,height,source})
}
}
/// `30000/1001`-style or display strings are irrelevant here: the first WxH on a `Video:` line is the
/// coded picture size, which is the aspect the scale+pad filter letterboxes into the canvas.
fn parse_video_size(line:&str)->Option<(u32,u32)> {
let rest=line.split_once("Video:")?.1;
rest.split(|c:char|c==','||c==' '||c=='('||c==')'||c=='['||c==']').find_map(|token|{
let token=token.trim_matches(|c:char|!c.is_ascii_digit()&&c!='x');
let (w,h)=token.split_once('x')?;
if w.is_empty()||h.is_empty()||!w.bytes().all(|b|b.is_ascii_digit())||!h.bytes().all(|b|b.is_ascii_digit()) {return None}
let (w,h)=(w.parse::<u32>().ok()?,h.parse::<u32>().ok()?);
((16..=16384).contains(&w)&&(16..=16384).contains(&h)).then_some((w,h))
})
}
/// FFmpeg ≥ 5.1 has `-fps_mode`; FFmpeg 8+ removed `-vsync`, and Ubuntu 22.04's 4.4 lacks `-fps_mode`.
/// Probe once per binary with a one-frame null run instead of parsing version strings.
fn passthrough_args(ffmpeg:&str)->(&'static str,&'static str) {
@@ -75,7 +102,11 @@ impl Decoder for Cli {
let mut out=Vec::new();
loop {
match self.frames.try_recv() {
Ok(rgb)=>out.push(Frame{width:self.width,height:self.height,rgb}),
Ok(rgb)=>{
let source=self.source.load(Ordering::Relaxed);
let content=(source!=0).then(||((source>>32) as u32,source as u32));
out.push(Frame{width:self.width,height:self.height,rgb,content})
},
Err(TryRecvError::Empty)=>break,
Err(TryRecvError::Disconnected)=>return Err("ffmpeg decoder exited".into()),
}
@@ -115,7 +146,7 @@ impl Drop for Cli { fn drop(&mut self) { let _=self.child.kill(); let _=self.chi
// Exercise the same streaming process and RGB conversion as SDL, then drain at EOF.
let mut frames=cli.decode(data).unwrap();drop(cli.input.take());
loop {match cli.frames.recv_timeout(std::time::Duration::from_secs(5)) {
Ok(rgb)=>frames.push(Frame{width:32,height:32,rgb}),
Ok(rgb)=>frames.push(Frame{width:32,height:32,rgb,content:None}),
Err(std::sync::mpsc::RecvTimeoutError::Disconnected)=>break,
Err(e)=>panic!("{codec:?} decoder did not finish: {e}"),
}}
@@ -128,4 +159,11 @@ impl Drop for Cli { fn drop(&mut self) { let _=self.child.kill(); let _=self.chi
}
}
}
#[test] fn source_size_is_read_from_the_ffmpeg_stream_line(){
assert_eq!(super::parse_video_size(" Stream #0:0: Video: h264 (High), yuv420p(progressive), 3024x1964, 25 fps"),Some((3024,1964)));
assert_eq!(super::parse_video_size("Stream #0:0: Video: hevc, yuv420p, 1920x1080 [SAR 1:1 DAR 16:9]"),Some((1920,1080)));
assert_eq!(super::parse_video_size("Stream #0:0: Audio: opus, 48000 Hz, stereo"),None);
assert_eq!(super::parse_video_size("frame= 1 fps=0.0 q=0.0"),None);
assert_eq!(super::parse_video_size("Stream #0:0: Video: h264, 1920xsynth"),None);
}
}
+3 -2
View File
@@ -5,7 +5,8 @@ pub struct Desktop { pub units:Latest<super::media::AccessUnit>, pub pli:Arc<Ato
impl Default for Desktop {
fn default()->Self {Self{units:Latest::new(3),pli:Arc::new(AtomicBool::new(false)),reset:Arc::new(AtomicBool::new(false)),ssrc:Arc::new(AtomicU32::new(0))}}
}
pub struct Frame { pub width:u32, pub height:u32, pub rgb:Vec<u8> }
/// `content` is the source picture size inside the (possibly padded) RGB canvas; None = full frame.
pub struct Frame { pub width:u32, pub height:u32, pub rgb:Vec<u8>, pub content:Option<(u32,u32)> }
pub trait Decoder { fn set_codec(&mut self,codec:super::media::Codec)->Result<(),String>; fn decode(&mut self, au:&[u8])->Result<Vec<Frame>,String>; fn reset(&mut self); }
pub trait FrameSink { fn present(&mut self,frame:&Frame)->Result<(),String>; }
/// Counts only (access units, presented frames, decode errors) for the periodic stats line.
@@ -29,7 +30,7 @@ pub fn display(decoder:&mut dyn Decoder,sink:&mut dyn FrameSink,au:&[u8],codec:s
}
#[cfg(test)] mod tests {
use super::*;
struct Mock(bool);impl Decoder for Mock {fn set_codec(&mut self,_:crate::viewer::media::Codec)->Result<(),String>{Ok(())} fn decode(&mut self,_:&[u8])->Result<Vec<Frame>,String>{if self.0{Err("synthetic".into())}else{Ok(vec![Frame{width:1,height:1,rgb:vec![0;3]}])}} fn reset(&mut self){self.0=false;}}
struct Mock(bool);impl Decoder for Mock {fn set_codec(&mut self,_:crate::viewer::media::Codec)->Result<(),String>{Ok(())} fn decode(&mut self,_:&[u8])->Result<Vec<Frame>,String>{if self.0{Err("synthetic".into())}else{Ok(vec![Frame{width:1,height:1,rgb:vec![0;3],content:None}])}} fn reset(&mut self){self.0=false;}}
struct Sink(usize);impl FrameSink for Sink{fn present(&mut self,f:&Frame)->Result<(),String>{assert_eq!(f.rgb.len(),3);self.0+=1;Ok(())}}
#[test] fn decode_error_requests_pli_and_recovers(){let mut d=Mock(true);let mut s=Sink(0);let p=AtomicBool::new(false);display(&mut d,&mut s,&[],crate::viewer::media::Codec::H264,&p).unwrap();assert!(p.load(Ordering::Acquire));display(&mut d,&mut s,&[],crate::viewer::media::Codec::H264,&p).unwrap();assert_eq!(s.0,1);}
}
+43
View File
@@ -35,6 +35,15 @@ pub fn mac_keycode(scan:i32)->Option<i32>{
_=>return None,
})
}
/// The content rectangle inside a decoder canvas padded to a fixed size (ffmpeg's scale+pad).
/// Mirrors `force_original_aspect_ratio=decrease,pad=…:(ow-iw)/2:(oh-ih)/2` so pointer input
/// ignores the padding instead of compressing the valid area.
pub fn content_rect(canvas:(u32,u32),content:(u32,u32))->Option<(f64,f64,f64,f64)> {
if canvas.0==0||canvas.1==0||content.0==0||content.1==0 {return None}
let scale=(canvas.0 as f64/content.0 as f64).min(canvas.1 as f64/content.1 as f64);
let(w,h)=(content.0 as f64*scale,content.1 as f64*scale);
Some(((canvas.0 as f64-w)/2.,(canvas.1 as f64-h)/2.,w,h))
}
#[derive(Clone,Copy,Debug,PartialEq)]
pub struct Viewport { pub x:f64, pub y:f64, pub width:f64, pub height:f64 }
impl Viewport {
@@ -75,6 +84,14 @@ impl Input {
let changed=if down{self.buttons.insert(button)}else{self.buttons.remove(&button)};
changed.then(||json!({"action":if down{"mouse_press"}else{"mouse_release"},"button":button}))
}
/// SDL wheel units are notches; the wire uses the Mac convention (one notch = 120).
/// Positive y is wheel-up, matching the host's `mouse_scroll` (button 4).
pub fn scroll(&mut self,x:f64,y:f64)->Option<Value> {
if self.view_only {return None}
let(dx,dy)=((x*120.).round() as i32,(y*120.).round() as i32);
if dx==0&&dy==0 {return None}
Some(json!({"action":"mouse_scroll","delta_x":dx,"delta_y":dy}))
}
pub fn release_all(&mut self)->Vec<Value> {
let mut out=Vec::new();
for key in std::mem::take(&mut self.keys).into_values().collect::<BTreeSet<_>>() {out.push(json!({"action":"kbd_release","key":key}));}
@@ -117,6 +134,15 @@ impl Input {
assert!(view_only(None,Some("1")).unwrap());assert!(view_only(Some("--view-only"),Some("0")).unwrap());
assert!(view_only(Some("invalid"),None).is_err());
}
#[test] fn wheel_notches_use_mac_wire_units(){
let mut i=Input::default();
assert_eq!(i.scroll(0.,1.).unwrap(),json!({"action":"mouse_scroll","delta_x":0,"delta_y":120}));
assert_eq!(i.scroll(0.,-1.).unwrap()["delta_y"],-120);
assert_eq!(i.scroll(1.,-2.).unwrap(),json!({"action":"mouse_scroll","delta_x":120,"delta_y":-240}));
assert_eq!(i.scroll(0.,0.1).unwrap()["delta_y"],12); // precise trackpad deltas
assert!(i.scroll(0.,0.).is_none());
assert!(Input::new(true).scroll(0.,1.).is_none());
}
#[test] fn mappings_encoding_and_focus() {
for (scan,vk) in [(4,65),(29,90),(39,48),(58,112),(80,37),(224,162),(230,165),(52,222)] {assert_eq!(windows_vk(scan),Some(vk));}
assert_eq!(windows_vk(100),None);
@@ -138,4 +164,21 @@ assert!(i.release_all().is_empty());
assert_eq!(v.motion(1010.,1000.).unwrap()["abs_x"],1.0);
assert!(v.motion(f64::NAN,0.).is_none());
}
#[test] fn padded_canvas_maps_only_its_content_rectangle(){
// 16:10 picture inside the default 16:9 canvas: 1728x1080 centred, 96 px pillarbox each side.
assert_eq!(content_rect((1920,1080),(1280,800)),Some((96.,0.,1728.,1080.)));
assert_eq!(content_rect((1920,1080),(1920,1080)),Some((0.,0.,1920.,1080.)));
// 4:3 picture: 1440x1080 centred, 240 px each side.
assert_eq!(content_rect((1920,1080),(640,480)),Some((240.,0.,1440.,1080.)));
assert_eq!(content_rect((0,1080),(640,480)),None);
// The content rectangle's edges are the screen edges, not the canvas edges.
let (x,y,w,h)=content_rect((1920,1080),(1280,800)).unwrap();
let v=Viewport::fit((1920,1080),(1280,800));
assert_eq!((x,y,w,h),(v.x,v.y,v.width,v.height));
let abs=|px:f64|v.motion(px,540.).unwrap();
assert_eq!(abs(96.)["abs_x"],0.0);
assert_eq!(abs(960.)["abs_x"],0.5);
assert_eq!(abs(1824.)["abs_x"],1.0);
assert_eq!(abs(0.)["abs_x"],0.0); // padding is clamped to the screen edge
}
}
+31 -3
View File
@@ -81,6 +81,11 @@ fn translate_actions(events:impl IntoIterator<Item=Event>,input:&mut Input,viewp
Event::MouseMotion{x,y,..}=>viewport.motion(x as f64,y as f64),
Event::MouseButtonDown{mouse_btn,..}=>button(mouse_btn).and_then(|b|input.button(b,true)),
Event::MouseButtonUp{mouse_btn,..}=>button(mouse_btn).and_then(|b|input.button(b,false)),
Event::MouseWheel{x,y,precise_x,precise_y,direction,..}=>{
let(dx,dy)=if precise_x!=0.||precise_y!=0.{(f64::from(precise_x),f64::from(precise_y))}else{(f64::from(x),f64::from(y))};
let(dx,dy)=match direction{sdl2::mouse::MouseWheelDirection::Flipped=>(-dx,-dy),_=>(dx,dy)};
input.scroll(dx,dy)
},
_=>None,
};if let Some(m)=message{messages.push(m);}
} (quit,messages)
@@ -89,12 +94,17 @@ fn translate_actions(events:impl IntoIterator<Item=Event>,input:&mut Input,viewp
fn button(b:MouseButton)->Option<i32>{match b{MouseButton::Left=>Some(1),MouseButton::Right=>Some(2),MouseButton::Middle=>Some(4),_=>None}}
impl FrameSink for Window {
fn present(&mut self,frame:&Frame)->Result<(),String>{
self.last_frame=Some(Frame{width:frame.width,height:frame.height,rgb:frame.rgb.clone()});
self.frame=(frame.width,frame.height);let v=self.viewport();let creator=self.canvas.texture_creator();
self.last_frame=Some(Frame{width:frame.width,height:frame.height,rgb:frame.rgb.clone(),content:frame.content});
// Pad bars belong to the decoded canvas, not the remote picture: viewport and input use the
// content aspect, and only the content sub-rectangle of the texture is drawn.
let content=frame.content.unwrap_or((frame.width,frame.height));
self.frame=content;let v=self.viewport();let creator=self.canvas.texture_creator();
let mut texture=creator.create_texture_streaming(PixelFormatEnum::RGB24,frame.width,frame.height).map_err(|_|"Texture creation failed")?;
texture.update(None,&frame.rgb,frame.width as usize*3).map_err(|_|"Texture update failed")?;
self.canvas.set_draw_color(sdl2::pixels::Color::BLACK);self.canvas.clear();
self.canvas.copy(&texture,None,Some(Rect::new(v.x as i32,v.y as i32,v.width as u32,v.height as u32)))?;self.canvas.present();Ok(())
let src=super::input::content_rect((frame.width,frame.height),content)
.map(|(x,y,w,h)|Rect::new(x.round() as i32,y.round() as i32,w.round().max(1.) as u32,h.round().max(1.) as u32));
self.canvas.copy(&texture,src,Some(Rect::new(v.x.round() as i32,v.y.round() as i32,v.width.round().max(1.) as u32,v.height.round().max(1.) as u32)))?;self.canvas.present();Ok(())
}
}
pub async fn run(session:&mut Session,window:&mut Window,decoder:&mut dyn super::desktop::Decoder,view_only:bool,device:&str)->Result<(),String>{
@@ -432,6 +442,24 @@ async fn pump_type_job(job: &mut TypeJob, session: &mut Session, window: &mut Wi
assert!(input.release_all().is_empty());
}
}
#[test] fn wheel_events_become_scroll_messages(){
use sdl2::mouse::MouseWheelDirection;
let mut input=Input::default();let mut focused=true;
let v=Viewport::fit((800,600),(1920,1080));
let wheel=|direction,x,y|Event::MouseWheel{timestamp:0,window_id:1,which:0,mouse_x:0,mouse_y:0,x,y,precise_x:0.,precise_y:0.,direction};
let(_,out)=translate([wheel(MouseWheelDirection::Normal,0,1),wheel(MouseWheelDirection::Flipped,0,1),wheel(MouseWheelDirection::Normal,0,0)],&mut input,v,&mut focused);
assert_eq!(out.len(),2);
assert_eq!(out[0]["action"],"mouse_scroll");
assert_eq!(out[0]["delta_y"],120);
assert_eq!(out[1]["delta_y"],-120);
// Trackpads report fractional precise deltas; a quarter notch is 30 wire units.
let precise=Event::MouseWheel{timestamp:0,window_id:1,which:0,mouse_x:0,mouse_y:0,x:0,y:0,precise_x:0.,precise_y:-0.25,direction:MouseWheelDirection::Normal};
let(_,out)=translate([precise],&mut input,v,&mut focused);
assert_eq!(out[0]["delta_y"],-30);
let mut watch=Input::new(true);let mut watching=true;
let(_,out)=translate([wheel(MouseWheelDirection::Normal,0,1)],&mut watch,v,&mut watching);
assert!(out.is_empty());
}
#[test] fn watch_events_never_emit_input_and_still_close(){
let mut input=Input::new(true);let mut focused=true;
assert!(input.key(4,true).is_none());assert!(input.button(1,true).is_none());assert!(!input.enabled());
+21 -11
View File
@@ -64,6 +64,9 @@ impl DesktopRole {
stream: Default::default(), stream_task: Default::default(), screen: Default::default(), tracking: Default::default(), request: Default::default() }
}
pub fn is_desktop(&self) -> bool { *self.active.borrow() }
/// A promoted peer whose video ended on its own (capture failed while locked or off, #114);
/// capture_desktop on it starts the video again.
fn stream_ended(&self) -> bool { self.stream_task.lock().unwrap().as_ref().is_some_and(|task| task.is_finished()) }
fn input(&self) -> Option<Arc<crate::input::DesktopInputSession>> { self.input.lock().unwrap().clone() }
fn screen_or(&self, screen: &Arc<ScreenCapture>) -> Arc<ScreenCapture> {
self.screen.lock().unwrap().clone().unwrap_or_else(|| screen.clone())
@@ -1316,10 +1319,14 @@ impl WebRtcSession {
}
});
} else if action == sa::CAPTURE_DESKTOP && channel_name == "CONTROL_DATA_CHANNEL" {
if message_role.is_desktop() {
if message_role.is_desktop() && !message_role.stream_ended() {
log::info!("SimpleAction 4 (capture_desktop) received on a desktop connection");
} else if let Some(request) = message_role.request.get() {
log::info!("SimpleAction 4 (capture_desktop) received on a tool connection; starting the desktop");
if message_role.is_desktop() {
log::info!("SimpleAction 4 (capture_desktop) received on a tool connection whose desktop video ended; restarting it");
} else {
log::info!("SimpleAction 4 (capture_desktop) received on a tool connection; starting the desktop");
}
request();
} else {
log::info!("SimpleAction 4 (capture_desktop) not handled: tool connection without a desktop route");
@@ -1461,18 +1468,21 @@ impl WebRtcSession {
/// Makes this tool peer the desktop: it takes the input lease, starts the video it negotiated
/// (H.264; audio was never negotiated) and repeats the desktop CONTROL initialization.
/// `screen` replaces a capture that was unavailable when the peer was offered.
/// A promoted peer whose video ended keeps its input lease and only restarts the video.
pub fn promote(&self, screen: Option<Arc<ScreenCapture>>) {
if self.role.is_desktop() { return; }
let restart = self.role.is_desktop();
if restart && !self.role.stream_ended() { return; }
if let Some(screen) = screen { *self.role.screen.lock().unwrap() = Some(screen); }
let session = self.input.desktop_session();
if self.pc.connection_state() == RTCPeerConnectionState::Connected { session.connected(); }
*self.role.input.lock().unwrap() = Some(session);
self.role.active.send_replace(true);
let stream = self.role.stream.lock().unwrap().clone();
if let Some(stream) = stream {
let task = stream.spawn(self.role.screen.lock().unwrap().clone(), self.role.active.subscribe());
if let Some(old) = std::mem::replace(&mut *self.role.stream_task.lock().unwrap(), task) { old.abort(); }
if !restart {
let session = self.input.desktop_session();
if self.pc.connection_state() == RTCPeerConnectionState::Connected { session.connected(); }
*self.role.input.lock().unwrap() = Some(session);
self.role.active.send_replace(true);
}
let stream = self.role.stream.lock().unwrap().clone();
let task = stream.and_then(|stream| stream.spawn(self.role.screen.lock().unwrap().clone(), self.role.active.subscribe()));
if let Some(old) = std::mem::replace(&mut *self.role.stream_task.lock().unwrap(), task) { old.abort(); }
if restart { log::info!("Desktop video restarted on a tool connection after its stream ended"); return; }
if !cfg!(test) { track_session(self.pc.clone(), self.actions.clone(), &self.role); }
log::info!("Desktop started on an existing tool connection");
}
@@ -121,13 +121,17 @@ impl PeerConnectionInternal {
ice_gather_policy: configuration.ice_transport_policy,
})?);
// Create the ice transport
pc.ice_transport = pc.create_ice_transport(api).await;
// Create the ice transport. Its state callback must read the real DTLS
// transport, which only exists after the ICE transport, so it is wired
// in through a weak slot (the placeholder would always report New).
let dtls_slot = Arc::new(std::sync::OnceLock::new());
pc.ice_transport = pc.create_ice_transport(api, Arc::clone(&dtls_slot)).await;
// Create the DTLS transport
let certificates = configuration.certificates.drain(..).collect();
pc.dtls_transport =
Arc::new(api.new_dtls_transport(Arc::clone(&pc.ice_transport), certificates)?);
let _ = dtls_slot.set(Arc::downgrade(&pc.dtls_transport));
// Create the SCTP transport
pc.sctp_transport = Arc::new(api.new_sctp_transport(Arc::clone(&pc.dtls_transport))?);
@@ -1106,13 +1110,16 @@ impl PeerConnectionInternal {
}
}
pub(super) async fn create_ice_transport(&self, api: &API) -> Arc<RTCIceTransport> {
pub(super) async fn create_ice_transport(
&self,
api: &API,
dtls_slot: Arc<std::sync::OnceLock<Weak<RTCDtlsTransport>>>,
) -> Arc<RTCIceTransport> {
let ice_transport = Arc::new(api.new_ice_transport(Arc::clone(&self.ice_gatherer)));
let ice_connection_state = Arc::clone(&self.ice_connection_state);
let peer_connection_state = Arc::clone(&self.peer_connection_state);
let is_closed = Arc::clone(&self.is_closed);
let dtls_transport = Arc::clone(&self.dtls_transport);
let on_ice_connection_state_change_handler =
Arc::clone(&self.on_ice_connection_state_change_handler);
let on_peer_connection_state_change_handler =
@@ -1139,7 +1146,11 @@ impl PeerConnectionInternal {
let on_peer_connection_state_change_handler2 =
Arc::clone(&on_peer_connection_state_change_handler);
let is_closed2 = Arc::clone(&is_closed);
let dtls_transport_state = dtls_transport.state();
let dtls_transport_state = dtls_slot
.get()
.and_then(Weak::upgrade)
.map(|dtls| dtls.state())
.unwrap_or(RTCDtlsTransportState::New);
let peer_connection_state2 = Arc::clone(&peer_connection_state);
Box::pin(async move {
RTCPeerConnection::do_ice_connection_state_change(