GNOME Portal:手机触控模式拖动按触摸屏重放,输入法 ASCII 直接按键输入 (#131)

* portal: mobile touch-mode drags become touch, consent asks once for it

In touch mode the Android viewer sends a finger as a left-button drag at
absolute positions (observed: press, mouse_move_absolute stream, release),
which on GNOME selects text instead of scrolling. SelectDevices now also
asks for the touchscreen type where AvailableDeviceTypes offers it, and a
drag from an Android/iOS controller (controller_platform 2/3) that follows
absolute motion is replayed as one finger through NotifyTouch* or EIS
touch, so the desktop scrolls and taps as on a touch screen. Relative
motion (the trackpad mode) and every other viewer keep mouse buttons.

A consent saved before this restores keyboard and pointer only. The first
mobile drag then asks the host once per run in the background (GNOME's
share dialog, also visible on the phone; the consent timeout closes it)
and swaps the new session in only when it grants the touchscreen for the
same monitor and size. The saved consent is never removed beforehand.
UU_TOUCH_INPUT=0 keeps mouse drags and never asks.

* input: type mobile IME text as portal keysyms on Latin GNOME layouts

Each mobile text_input was set as the clipboard and pasted with
Shift+Insert, which VTE terminals bind to the PRIMARY selection: text
typed from the phone into a terminal pasted whatever was last selected
(observed: a selected line of box-drawing characters for every letter).
When GNOME's active input source (first of mru-sources) is a Latin xkb
layout, printable ASCII, newline and tab now go through
NotifyKeyboardKeysym, where Mutter picks the key and Shift level of the
active layout. Other text, IBus engines, KDE and EIS sessions keep the
clipboard paste, which no longer replaces the clipboard for plain typing.

* portal test: accept libei before 1.3 rejecting a touch outside one of two regions

libei 1.2 (Ubuntu 24.04, CI) accepts a touch only inside every region of the
device; 1.3.0 checks any region. The harness device also carries a region for
another screen, so the older library rejects the touch with a client-bug
diagnostic. A real one-stream session has one region.
This commit is contained in:
Its.spixy authored and GitHub committed 2026-10-29 19:49:15 +08:00
1 parent 6a0cd5bda6
commit 11097b5847
13 files changed
+404 -51

No files matched your search

+186 -10
View File
@@ -46,12 +46,35 @@ struct HeldInput { buttons: std::collections::BTreeSet<i32>, keys: std::collecti
/// Held modifier VKs (bit 0 Ctrl, 1 Shift, 2 Alt, 3 Win) and keys that got a synthetic Shift.
#[derive(Default)]
struct Shortcuts { modifiers: u8, shifted: std::collections::BTreeSet<i32> }
/// Mobile viewers in touch mode send a finger as a left-button drag at absolute positions (observed:
/// Android sends press, a stream of mouse_move_absolute, release), which on a desktop selects text
/// instead of scrolling. With the portal touchscreen granted those drags are replayed as one finger,
/// so the desktop scrolls and taps as on a touch screen. `at` is the last absolute position; relative
/// motion (the viewers' trackpad mode) clears it and keeps mouse buttons.
#[derive(Default)]
struct TouchDrag { mobile: bool, at: Option<(i32,i32)>, down: bool }
/// Viewer platform from the connect header `controller_platform` (website enum: 1 windows, 2 android,
/// 3 ios, 4 macos, 5 tv).
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub enum Controller { #[default] Other, Mac, Mobile }
impl Controller {
pub fn from_platform(platform: Option<&serde_json::Value>) -> Self {
let Some(v) = platform else { return Self::Other };
let text = v.as_str().map(str::to_ascii_lowercase);
match v.as_i64().or_else(|| text.as_deref().and_then(|s| s.parse().ok())) {
Some(4) => Self::Mac,
Some(2 | 3) => Self::Mobile,
_ if text.is_some_and(|s| s.contains("mac") || s.contains("darwin")) => Self::Mac,
_ => Self::Other,
}
}
}
/// Mac viewer with 「使用 macOS 快捷键」 off sends Cmd as VK_LWIN (verified: Cmd 0x5B, Option 0xA4,
/// Ctrl 0xA2). Cmd is held back until we know whether it starts a Mac shortcut.
#[derive(Default)]
struct CmdState { held: bool, pending: bool, passthrough: bool, combos: std::collections::BTreeMap<i32, bool> }
pub struct InputInjector { mac_controller: std::sync::atomic::AtomicBool, cmd: std::sync::Mutex<CmdState>, serial:std::sync::Mutex<()>, owner:std::sync::Mutex<DesktopOwner>, shortcuts: std::sync::Mutex<Shortcuts>, uinput: Option<crate::uinput::Devices>, screen: std::sync::Mutex<(u32,u32)>, portal_failures: std::sync::atomic::AtomicUsize, portal_outcomes: [std::sync::atomic::AtomicUsize; 3], portal:std::sync::Mutex<Option<std::sync::Arc<crate::portal::PortalSession>>>, backend: crate::desktop_backend::DesktopBackend, pending: std::sync::Mutex<PendingPointer>, held: std::sync::Mutex<HeldInput> }
pub struct InputInjector { mac_controller: std::sync::atomic::AtomicBool, cmd: std::sync::Mutex<CmdState>, serial:std::sync::Mutex<()>, owner:std::sync::Mutex<DesktopOwner>, shortcuts: std::sync::Mutex<Shortcuts>, uinput: Option<crate::uinput::Devices>, screen: std::sync::Mutex<(u32,u32)>, portal_failures: std::sync::atomic::AtomicUsize, portal_outcomes: [std::sync::atomic::AtomicUsize; 3], portal:std::sync::Mutex<Option<std::sync::Arc<crate::portal::PortalSession>>>, backend: crate::desktop_backend::DesktopBackend, pending: std::sync::Mutex<PendingPointer>, held: std::sync::Mutex<HeldInput>, touch: std::sync::Mutex<TouchDrag> }
impl InputInjector {
pub fn new() -> Self {
@@ -62,7 +85,7 @@ impl InputInjector {
Err(e) => { log::warn!("uinput unavailable ({}); using the desktop input path. See README for the udev rule.", e.kind()); None }
}
} else { None };
Self { mac_controller: std::sync::atomic::AtomicBool::new(false), cmd: std::sync::Mutex::new(CmdState::default()), serial:std::sync::Mutex::new(()), owner:std::sync::Mutex::new(DesktopOwner::default()), shortcuts: std::sync::Mutex::new(Shortcuts::default()), uinput, screen: std::sync::Mutex::new((0,0)), portal_failures: std::sync::atomic::AtomicUsize::new(0), portal_outcomes: std::array::from_fn(|_| std::sync::atomic::AtomicUsize::new(0)), portal:std::sync::Mutex::new(None), backend:crate::desktop_backend::detect().unwrap_or(crate::desktop_backend::DesktopBackend::X11), pending: std::sync::Mutex::new(PendingPointer::default()), held:std::sync::Mutex::new(HeldInput::default()) }
Self { mac_controller: std::sync::atomic::AtomicBool::new(false), cmd: std::sync::Mutex::new(CmdState::default()), serial:std::sync::Mutex::new(()), owner:std::sync::Mutex::new(DesktopOwner::default()), shortcuts: std::sync::Mutex::new(Shortcuts::default()), uinput, screen: std::sync::Mutex::new((0,0)), portal_failures: std::sync::atomic::AtomicUsize::new(0), portal_outcomes: std::array::from_fn(|_| std::sync::atomic::AtomicUsize::new(0)), portal:std::sync::Mutex::new(None), backend:crate::desktop_backend::detect().unwrap_or(crate::desktop_backend::DesktopBackend::X11), pending: std::sync::Mutex::new(PendingPointer::default()), held:std::sync::Mutex::new(HeldInput::default()), touch:std::sync::Mutex::new(TouchDrag::default()) }
}
pub fn bind_portal(&self,session:Option<std::sync::Arc<crate::portal::PortalSession>>) {
@@ -114,8 +137,15 @@ impl InputInjector {
let mut pending = self.pending.lock().unwrap();
let motion = std::mem::take(&mut *pending);
pending.scroll = (motion.scroll.0 % 120, motion.scroll.1 % 120);
if let Some((x,y)) = motion.absolute { self.inject_move(x,y,false); }
if motion.relative != (0,0) { self.inject_move(motion.relative.0 as i32,motion.relative.1 as i32,true); }
if let Some((x,y)) = motion.absolute {
let mut touch = self.touch.lock().unwrap();
if touch.down { self.inject_touch(1,x,y); } else { self.inject_move(x,y,false); }
touch.at = Some((x,y));
}
if motion.relative != (0,0) {
self.touch.lock().unwrap().at = None;
self.inject_move(motion.relative.0 as i32,motion.relative.1 as i32,true);
}
for (value, positive, negative) in [(motion.scroll.1,4,5),(motion.scroll.0,7,6)] {
let count = value / 120;
let button = if count > 0 { positive } else { negative };
@@ -136,6 +166,7 @@ impl InputInjector {
pub fn mouse_button(&self, button: i32, is_press: bool) {
let _serial=self.serial.lock().unwrap();
self.flush_pointer_locked();
if button == 1 && self.touch_button(is_press) { return; }
let result = self.inject_button(button,is_press);
if result == 0 {
let mut held = self.held.lock().unwrap();
@@ -174,10 +205,52 @@ impl InputInjector {
shift
}
/// The current desktop controller is the macOS viewer (connect header controller_platform).
pub fn set_mac_controller(&self, mac: bool) {
self.mac_controller.store(mac, std::sync::atomic::Ordering::Relaxed);
/// The current desktop controller's platform (connect header controller_platform).
/// `UU_TOUCH_INPUT=0` keeps mobile drags as mouse drags.
pub fn set_controller(&self, controller: Controller) {
self.mac_controller.store(controller == Controller::Mac, std::sync::atomic::Ordering::Relaxed);
*self.cmd.lock().unwrap() = CmdState::default();
let mut touch = self.touch.lock().unwrap();
if touch.down { self.inject_touch(2,0,0); }
*touch = TouchDrag { mobile: controller == Controller::Mobile && std::env::var("UU_TOUCH_INPUT").as_deref() != Ok("0"), ..TouchDrag::default() };
}
/// Left button of a mobile controller: a finger down at the last absolute position, or up.
/// False leaves the button to the mouse path (no touchscreen, trackpad mode, other viewers).
fn touch_button(&self, pressed: bool) -> bool {
let mut touch = self.touch.lock().unwrap();
if !pressed {
if !touch.down { return false; }
touch.down = false;
self.inject_touch(2,0,0);
return true;
}
if touch.down { return true; }
let Some((x,y)) = touch.at.filter(|_| touch.mobile) else { return false };
let supported = self.backend == crate::desktop_backend::DesktopBackend::Portal
&& self.portal.lock().unwrap().as_ref().is_some_and(|portal| portal.touch_supported());
static REPORTED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
if !REPORTED.swap(true, std::sync::atomic::Ordering::Relaxed) {
log::info!("Mobile controller drag input: touchscreen={supported}");
}
if !supported { self.offer_touch_consent(); }
touch.down = supported && self.inject_touch(0,x,y) == 0;
touch.down
}
/// The portal offers a touchscreen the restored consent lacks: ask the host once per run, in the
/// background (GNOME's share dialog, visible to the phone too); drags stay mouse drags meanwhile.
fn offer_touch_consent(&self) {
static ASKED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(false);
let Some(portal) = self.portal.lock().unwrap().clone() else { return };
if ASKED.swap(true, std::sync::atomic::Ordering::Relaxed) { return }
std::thread::spawn(move || match portal.upgrade_touch() {
Ok(true) => log::info!("Touch consent upgraded: mobile drags are touch from now on"),
Ok(false) => log::info!("Touch consent upgrade not needed or not offered by the portal"),
Err(reason) => log::info!("Touch consent upgrade not granted: {reason}"),
});
}
fn inject_touch(&self, phase: i32, x: i32, y: i32) -> i32 {
let Some(portal) = self.portal.lock().unwrap().clone() else { return -1 };
self.portal_result("touch", portal.touch(phase, f64::from(x), f64::from(y)))
}
fn terminal_focused(&self) -> bool {
@@ -269,6 +342,7 @@ impl InputInjector {
pub fn release_held(&self) {
let _serial=self.serial.lock().unwrap();
self.discard_pointer();
{ let mut touch=self.touch.lock().unwrap(); if std::mem::take(&mut touch.down) { self.inject_touch(2,0,0); } }
let mut held=self.held.lock().unwrap();
if let Some(uinput)=&self.uinput {
Self::uinput_result(uinput.release_all());
@@ -494,8 +568,16 @@ impl InputInjector {
}
} else if self.backend==crate::desktop_backend::DesktopBackend::Portal {
if std::env::var("UU_WAYLAND_TEXT").is_ok_and(|v|v=="0") { return; }
if self.paste_text_via_clipboard(text) {
log::info!("Desktop text input (Portal clipboard paste): chars={} result=ok", text.chars().count());
// Shift+Insert pastes the PRIMARY selection in VTE terminals, not the clipboard just set:
// type what the active layout has as keys and paste only the rest.
let typed=self.type_portal_keysyms(text);
if typed==text.chars().count() {
log::info!("Desktop text input (Portal keysyms): chars={typed}");
return;
}
let rest:String=text.chars().skip(typed).collect();
if self.paste_text_via_clipboard(&rest) {
log::info!("Desktop text input (Portal clipboard paste): chars={} typed={typed} result=ok", text.chars().count());
return;
}
static WARNED:std::sync::atomic::AtomicBool=std::sync::atomic::AtomicBool::new(false);
@@ -509,6 +591,14 @@ impl InputInjector {
}
}
}
/// Types `text` as portal keysyms when it is all printable ASCII (or newline/tab) and GNOME's
/// active input source is a Latin xkb layout; returns how many characters were typed.
fn type_portal_keysyms(&self, text: &str) -> usize {
let Some(symbols)=ascii_keysyms(text) else { return 0 };
if !gnome_layout_types_ascii() { return 0; }
let Some(portal)=self.portal.lock().unwrap().clone() else { return 0 };
symbols.iter().take_while(|&&symbol| portal.type_keysym(symbol).is_ok()).count()
}
pub fn key_event(&self, keycode: i32, is_press: bool) {
let _serial=self.serial.lock().unwrap();
if self.backend!=crate::desktop_backend::DesktopBackend::X11 { return; }
@@ -645,7 +735,93 @@ impl InputInjector {
uinput:None, screen:std::sync::Mutex::new((0,0)), portal_failures:std::sync::atomic::AtomicUsize::new(0),
portal_outcomes:std::array::from_fn(|_|std::sync::atomic::AtomicUsize::new(0)),
portal:std::sync::Mutex::new(None), backend:crate::desktop_backend::DesktopBackend::Portal,
pending:std::sync::Mutex::new(PendingPointer::default()), held:std::sync::Mutex::new(HeldInput::default()) }
pending:std::sync::Mutex::new(PendingPointer::default()), held:std::sync::Mutex::new(HeldInput::default()),
touch:std::sync::Mutex::new(TouchDrag::default()) }
}
}
/// Keysyms for mobile IME text: printable ASCII is its own keysym, newline is Return, tab is Tab.
fn ascii_keysyms(text: &str) -> Option<Vec<u32>> {
text.chars().map(|c| match c { '\n' => Some(0xff0d), '\t' => Some(0xff09), ' '..='~' => Some(c as u32), _ => None }).collect()
}
/// Whether an `org.gnome.desktop.input-sources` source list starts with an xkb layout that has
/// printable ASCII in its first group. Mutter drops keysyms the active layout lacks, and an IBus
/// engine (pinyin, ...) would compose the keys instead.
fn first_source_types_ascii(sources: &str) -> bool {
let Some(start)=sources.find('(') else { return false };
let Some(end)=sources[start..].find(')') else { return false };
let fields:Vec<&str>=sources[start+1..start+end].split(',').map(|f| f.trim().trim_matches('\'')).collect();
let [kind, layout]=fields[..] else { return false };
let (base, variant)=layout.split_once('+').unwrap_or((layout, ""));
const NON_LATIN: &[&str]=&["af","am","ara","bd","bg","bt","by","eg","et","ge","gr","il","in","iq","ir","kg","kh","kz","la","lk","mk","mm","mn","mv","np","ru","sy","th","tj","ua"];
kind=="xkb" && !NON_LATIN.contains(&base) && !variant.contains("cyrillic") && (base!="rs" || variant.contains("latin"))
}
/// GNOME's active input source is the first of `mru-sources` (else of `sources`). Cached for 2 s.
fn gnome_layout_types_ascii() -> bool {
#[cfg(test)]
{ false }
#[cfg(not(test))]
{
static CACHE: std::sync::Mutex<Option<(std::time::Instant,bool)>> = std::sync::Mutex::new(None);
let mut cache=CACHE.lock().unwrap();
if let Some((at,value))=*cache { if at.elapsed()<std::time::Duration::from_secs(2) { return value; } }
let read=|key: &str| std::process::Command::new("gsettings").args(["get","org.gnome.desktop.input-sources",key])
.stderr(std::process::Stdio::null()).output().ok().filter(|o| o.status.success())
.map(|o| String::from_utf8_lossy(&o.stdout).into_owned()).filter(|s| s.contains('('));
let value=read("mru-sources").or_else(|| read("sources")).is_some_and(|s| first_source_types_ascii(&s));
*cache=Some((std::time::Instant::now(),value));
value
}
}
#[cfg(test)]
mod text_keysym_tests {
use super::*;
#[test]
fn ascii_text_becomes_keysyms_and_other_text_does_not() {
assert_eq!(ascii_keysyms("Hi!\n\t~"), Some(vec![0x48,0x69,0x21,0xff0d,0xff09,0x7e]));
assert_eq!(ascii_keysyms("čaj"), None);
assert_eq!(ascii_keysyms("测试"), None);
}
#[test]
fn only_latin_xkb_layouts_type_ascii() {
assert!(first_source_types_ascii("[('xkb', 'us'), ('ibus', 'libpinyin'), ('xkb', 'rs+latin')]"));
assert!(first_source_types_ascii("[('xkb', 'rs+latin')]"));
assert!(first_source_types_ascii("[('xkb', 'de+nodeadkeys')]"));
assert!(!first_source_types_ascii("[('ibus', 'libpinyin'), ('xkb', 'us')]"));
assert!(!first_source_types_ascii("[('xkb', 'rs')]"));
assert!(!first_source_types_ascii("[('xkb', 'ru')]"));
assert!(first_source_types_ascii("[('xkb', 'ba+unicodeus')]"));
assert!(!first_source_types_ascii("[('xkb', 'me+cyrillic')]"));
assert!(!first_source_types_ascii("@a(ss) []"));
}
}
#[cfg(test)]
mod touch_tests {
use super::*;
#[test]
fn controller_from_platform_numbers_and_mac_strings() {
use serde_json::json;
assert_eq!(Controller::from_platform(Some(&json!(2))), Controller::Mobile);
assert_eq!(Controller::from_platform(Some(&json!("3"))), Controller::Mobile);
assert_eq!(Controller::from_platform(Some(&json!(4))), Controller::Mac);
assert_eq!(Controller::from_platform(Some(&json!("MacOS"))), Controller::Mac);
assert_eq!(Controller::from_platform(Some(&json!(1))), Controller::Other);
assert_eq!(Controller::from_platform(None), Controller::Other);
}
#[test]
fn mobile_drag_without_touchscreen_stays_a_mouse_button() {
let input=InputInjector::without_desktop();
input.set_controller(Controller::Mobile);
input.mouse_absolute_normalized(0.5,0.5,(100,100)); input.flush_pointer();
assert!(input.touch.lock().unwrap().at.is_some());
assert!(!input.touch_button(true) && !input.touch.lock().unwrap().down);
assert!(!input.touch_button(false));
input.mouse_relative(1,1); input.flush_pointer();
assert!(input.touch.lock().unwrap().at.is_none());
input.set_controller(Controller::Other);
assert!(!input.touch.lock().unwrap().mobile);
}
}
+10 -10
View File
@@ -763,8 +763,8 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
// One peer per viewer connection (desktop, file transfer, terminal...), keyed by ice_id.
let mut next_capture_type: Option<(u64,crate::session_ice::SessionIce)> = None;
let mut capture_types: HashMap<String, (u64,crate::session_ice::SessionIce)> = HashMap::new();
let mut mac_by_ice: HashMap<String, bool> = HashMap::new();
let mut next_mac = false;
let mut controller_by_ice: HashMap<String, crate::input::Controller> = HashMap::new();
let mut next_controller = crate::input::Controller::Other;
// Tool peers ask here to become the desktop (capture_desktop on an existing connection, #99).
let (desktop_requests, mut desktop_requested) = mpsc::unbounded_channel::<String>();
@@ -951,9 +951,9 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
.and_then(|s| s.as_str())
);
}
SignalingEvent::ConnectRequest { capture_type, ice_id, transport, controller_mac } => {
if mac_by_ice.len()>=32 { mac_by_ice.clear(); }
match ice_id.clone() { Some(id) => { mac_by_ice.insert(id, controller_mac); } None => next_mac = controller_mac }
SignalingEvent::ConnectRequest { capture_type, ice_id, transport, controller } => {
if controller_by_ice.len()>=32 { controller_by_ice.clear(); }
match ice_id.clone() { Some(id) => { controller_by_ice.insert(id, controller); } None => next_controller = controller }
// Pending cloud credentials live only within this room; cap
// requests that never receive an offer rather than retain them.
if capture_types.len()>=32 { capture_types.clear(); }
@@ -984,8 +984,8 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
let (capture_type,transport) = capture_types.remove(&ice).or_else(|| next_capture_type.take()).unwrap_or((1,crate::session_ice::SessionIce::default()));
// CaptureType: 0 unknown, 1 desktop, 2 window; 5 file transfer, 8 terminal, 9 port mapping.
let desktop = capture_type <= 2;
let controller_mac = mac_by_ice.remove(&ice).unwrap_or(next_mac);
if desktop { input.set_mac_controller(controller_mac); }
let controller = controller_by_ice.remove(&ice).unwrap_or(next_controller);
if desktop { input.set_controller(controller); }
log::info!("Offer for connection: capture_type={} desktop={} active_peers={}", capture_type, desktop, sessions.len());
// An expired NVENC failure is re-probed in the background. This offer keeps the cached encoder.
if desktop { video::warm_auto_probe(); video::check_ffmpeg_usable(); }
@@ -1017,7 +1017,7 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
match WebRtcSession::new(session_capture, input.clone(), signaling_client.clone(), desktop, transport,
if desktop { portal_desktop.clone() } else { None }).await {
Ok(mut sess) => {
sess.controller_mac = controller_mac;
sess.controller = controller;
if !desktop {
let (requests, peer) = (desktop_requests.clone(), ice.clone());
sess.on_desktop_request(move || { let _ = requests.send(peer.clone()); });
@@ -1084,7 +1084,7 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
log::info!("Desktop request on a tool connection not supported with the Portal backend; open the desktop before port mapping");
continue;
}
let controller_mac = peer.controller_mac;
let controller = peer.controller;
let current = peer.desktop_capture();
// Same recovery as a desktop offer: the display may have come up or been resized after this peer.
let resized = current.init_error.is_none() && current.backend == desktop_backend::DesktopBackend::X11
@@ -1098,7 +1098,7 @@ async fn serve(mut capture: Arc<ScreenCapture>, input: Arc<InputInjector>, shutd
}
} else { None };
video::warm_auto_probe(); video::check_ffmpeg_usable();
input.set_mac_controller(controller_mac);
input.set_controller(controller);
input.bind_portal(screen.as_ref().map_or_else(|| sessions[&ice].capture.portal.clone(), |screen| screen.portal.clone()));
// One desktop at a time, as for a new desktop offer. Peers offered as desktops
// close; a promoted tool peer only stops its desktop and keeps its tools.
+51 -6
View File
@@ -38,6 +38,9 @@ mod native {
fn uu_portal_move(portal: *mut c_void, x: f64, y: f64, relative: i32) -> i32;
fn uu_portal_button(portal: *mut c_void, button: i32, pressed: i32) -> i32;
fn uu_portal_key(portal: *mut c_void, keysym: u32, keycode: i32, pressed: i32) -> i32;
fn uu_portal_touch_supported(portal: *mut c_void) -> i32;
fn uu_portal_touch_available(portal: *mut c_void) -> i32;
fn uu_portal_touch(portal: *mut c_void, phase: i32, x: f64, y: f64) -> i32;
fn uu_portal_stream_position(portal: *mut c_void, x: *mut i32, y: *mut i32) -> i32;
fn uu_portal_monitors(out: *mut libc::c_char, length: usize) -> i32;
fn uu_portal_end(portal: *mut c_void);
@@ -114,7 +117,8 @@ mod native {
}
/// Opens a session for `monitor` (a Mutter connector) with that monitor's saved consent;
/// None means the screen currently selected, or the only monitor.
fn open_inner(cancelled: Option<&std::sync::atomic::AtomicBool>, monitor: Option<&str>) -> Result<Self, super::PortalError> {
/// `fresh` asks the host again instead of restoring the saved consent (which stays saved).
fn open_inner(cancelled: Option<&std::sync::atomic::AtomicBool>, monitor: Option<&str>, fresh: bool) -> Result<Self, super::PortalError> {
extern "C" fn is_cancelled(data: *mut c_void) -> i32 {
let cancelled = unsafe { &*(data as *const std::sync::atomic::AtomicBool) };
i32::from(cancelled.load(std::sync::atomic::Ordering::Acquire))
@@ -129,7 +133,7 @@ mod native {
let key = super::permissions::desktop_key();
let wanted = monitor.map(str::to_owned).or_else(crate::wayland::portal_monitor);
// Each monitor has its own consent; the desktop-wide key is the single-monitor one.
let token = directory.as_deref().and_then(|dir| {
let token = directory.as_deref().filter(|_| !fresh).and_then(|dir| {
wanted.as_deref().and_then(|name| super::permissions::take(dir, &super::permissions::monitor_key(name)))
.or_else(|| super::permissions::take(dir, key))
}).and_then(|value| std::ffi::CString::new(value).ok());
@@ -188,7 +192,7 @@ mod native {
/// time GNOME asks on the host) and swap it in, so every holder of this Arc follows.
pub fn switch_to(&self, connector: &str) -> Result<(), String> {
if self.monitor().as_deref() == Some(connector) { return Ok(()); }
let replacement = Self::open_inner(None, Some(connector)).map_err(|error| error.to_string())?;
let replacement = Self::open_inner(None, Some(connector), false).map_err(|error| error.to_string())?;
if replacement.monitor().as_deref() != Some(connector) {
return Err("Another monitor was shared; choose this monitor in the host dialog".into());
}
@@ -204,10 +208,29 @@ mod native {
Ok(())
}
pub fn open() -> Result<Self, String> {
Self::open_inner(None, None).map_err(|error| error.to_string())
Self::open_inner(None, None, false).map_err(|error| error.to_string())
}
/// A consent saved before touch support restores keyboard and pointer only. Ask the host once
/// more (GNOME shows its share dialog; the consent timeout closes it) and swap the new session
/// in when it grants the touchscreen for this monitor at this size; until then, and on any
/// refusal, this session and its saved consent stay as they are. Ok(false): nothing to do.
pub fn upgrade_touch(&self) -> Result<bool, String> {
let available = self.handle.lock().is_ok_and(|handle| unsafe { uu_portal_touch_available(handle.0.as_ptr()) } != 0);
if !available || self.touch_supported() { return Ok(false); }
log::info!("Portal touch consent: asking the host again for the touchscreen");
let monitor = self.monitor();
let replacement = Self::open_inner(None, monitor.as_deref(), true).map_err(|error| error.to_string())?;
// The new consent is saved for whichever monitor was granted, so a later connection uses it.
if replacement.monitor() != monitor { return Err("another monitor was shared".into()); }
if !replacement.touch_supported() { return Err("the touchscreen was not granted".into()); }
self.check_geometry(&replacement).map_err(|error| error.to_string())?;
let next = replacement.handle.into_inner().map_err(|_| "Portal session lock poisoned")?;
let previous = std::mem::replace(&mut *self.handle.lock().map_err(|_| "Portal session lock poisoned")?, next);
drop(previous);
Ok(true)
}
pub fn open_cancellable(cancelled: &std::sync::atomic::AtomicBool) -> Result<Self, super::PortalError> {
Self::open_inner(Some(cancelled), None)
Self::open_inner(Some(cancelled), None, false)
}
pub fn recover(&self, cancelled: &std::sync::atomic::AtomicBool) -> Result<bool, super::PortalError> {
use std::sync::atomic::Ordering;
@@ -305,7 +328,7 @@ mod native {
self.was_locked.store(true, std::sync::atomic::Ordering::Release);
false
}) { return Ok(None); }
let replacement = match Self::open_inner(None, self.monitor().as_deref()) {
let replacement = match Self::open_inner(None, self.monitor().as_deref(), false) {
Ok(value) => value,
Err(error) => {
log::warn!("Portal capture recovery deferred: {error}");
@@ -372,6 +395,24 @@ mod native {
let handle = self.handle.lock().map_err(|_| "Portal session lock poisoned")?;
super::check(unsafe { uu_portal_button(handle.0.as_ptr(), button, i32::from(pressed)) })
}
/// Press and release a keysym through Notify*: the compositor picks the key and level of
/// the active layout. Under EIS (keycodes only) it fails before typing anything.
pub fn type_keysym(&self, symbol: u32) -> Result<(), String> {
if *self.locked.borrow() != Some(false) { return Err("Portal input blocked while locked".into()); }
let handle = self.handle.lock().map_err(|_| "Portal session lock poisoned")?;
super::check(unsafe { uu_portal_key(handle.0.as_ptr(), symbol, -1, 1) })?;
super::check(unsafe { uu_portal_key(handle.0.as_ptr(), symbol, -1, 0) })
}
/// The portal granted the touchscreen device type (requested where available).
pub fn touch_supported(&self) -> bool {
self.handle.lock().is_ok_and(|handle| unsafe { uu_portal_touch_supported(handle.0.as_ptr()) } != 0)
}
/// One finger in stream pixels: phase 0 down, 1 motion, 2 up.
pub fn touch(&self, phase: i32, x: f64, y: f64) -> Result<(), String> {
if phase != 2 && *self.locked.borrow() != Some(false) { return Err("Portal input blocked while locked".into()); }
let handle = self.handle.lock().map_err(|_| "Portal session lock poisoned")?;
super::check(unsafe { uu_portal_touch(handle.0.as_ptr(), phase, x, y) })
}
pub fn key(&self, symbol: u64, pressed: bool) -> Result<(), String> {
if pressed && *self.locked.borrow() != Some(false) { return Err("Portal input blocked while locked".into()); }
let symbol = u32::try_from(symbol).map_err(|_| "Invalid Portal keysym")?;
@@ -693,6 +734,10 @@ impl PortalSession {
pub fn capture_changed(&self, _: &mut [u8], _: &mut Option<FrameVersion>) -> Result<bool, String> { Err(error(15)) }
pub fn move_pointer(&self, _: f64, _: f64, _: bool) -> Result<(), String> { Err(error(15)) }
pub fn button(&self, _: i32, _: bool) -> Result<(), String> { Err(error(15)) }
pub fn touch_supported(&self) -> bool { false }
pub fn upgrade_touch(&self) -> Result<bool, String> { Err(error(15)) }
pub fn type_keysym(&self, _: u32) -> Result<(), String> { Err(error(15)) }
pub fn touch(&self, _: i32, _: f64, _: f64) -> Result<(), String> { Err(error(15)) }
pub fn key(&self, _: u64, _: bool) -> Result<(), String> { Err(error(15)) }
}
+4 -5
View File
@@ -51,7 +51,7 @@ pub enum SignalingEvent {
BeControlled(Value),
/// be-controlled with its binary app_data (ConnectOptions): capture type and the
/// ice_id it announces, when present.
ConnectRequest { capture_type: u64, ice_id: Option<String>, transport:crate::session_ice::SessionIce, controller_mac: bool },
ConnectRequest { capture_type: u64, ice_id: Option<String>, transport:crate::session_ice::SessionIce, controller: crate::input::Controller },
Soac(SoacMessage),
Released(Value),
Left(Value),
@@ -513,10 +513,9 @@ impl SignalingClient {
if !caps.is_empty() { log::info!("Viewer decoder caps (fps,codec,w,h,chroma fields): {:?}", caps); }
// Viewer platform (website enum: 1 windows, 2 android, 3 ios, 4 macos, 5 tv).
let platform = header.get("controller_platform").or_else(|| header.get("data").and_then(|d| d.get("controller_platform")));
let controller_mac = platform.is_some_and(|v| v.as_i64() == Some(4)
|| v.as_str().is_some_and(|s| s == "4" || s.to_ascii_lowercase().contains("mac") || s.to_ascii_lowercase().contains("darwin")));
log::info!("Controller platform: {} mac={}", platform.map(|v| v.to_string()).unwrap_or_default().chars().take(24).collect::<String>(), controller_mac);
if !send_event(&event_sender, SignalingEvent::ConnectRequest { capture_type, ice_id, transport:crate::session_ice::SessionIce::from_header(&header), controller_mac }).await { break; }
let controller = crate::input::Controller::from_platform(platform);
log::info!("Controller platform: {} controller={:?}", platform.map(|v| v.to_string()).unwrap_or_default().chars().take(24).collect::<String>(), controller);
if !send_event(&event_sender, SignalingEvent::ConnectRequest { capture_type, ice_id, transport:crate::session_ice::SessionIce::from_header(&header), controller }).await { break; }
}
}
}
+2 -2
View File
@@ -111,7 +111,7 @@ pub struct WebRtcSession {
/// The role this peer was offered with; `role` also covers a later capture_desktop.
pub desktop: bool,
pub role: Arc<DesktopRole>,
pub controller_mac: bool,
pub controller: crate::input::Controller,
pub context: Arc<Mutex<SessionContext>>,
actions: Arc<crate::system_actions::PeerActions>,
}
@@ -1454,7 +1454,7 @@ impl WebRtcSession {
input,
desktop,
role,
controller_mac: false,
controller: crate::input::Controller::Other,
context,
actions: session_actions,
})