* feat(portal): follow the viewer's mouse mode with the embedded cursor
The portal stream hides the compositor cursor. A viewer that drives the pointer
with relative motion (UU's controlled-end mouse, also the Mac trackpad mode)
has no absolute position to draw its own pointer at, and the game's cursor
never reaches the picture, so the mouse can be moved and clicked but not seen.
Request cursor_mode=embedded while relative motion arrives, and keep it hidden
while absolute motion arrives: there the viewer draws its own pointer and an
embedded copy would trail it. The portal reads cursor_mode only when the
session is created, so switch by rebuilding the session with the saved consent,
the same path the privacy screen uses. A run has to last five events and 200 ms
before it counts, so a burst at connect or an alternating controller does not
rebuild; a failed rebuild backs off for five seconds.
The rebuilt session is checked against the monitor the previous one captured,
so a restored consent cannot silently move the share. The engine keeps the mode
for later opens (recovery, privacy, monitor switches) and skips the switch when
an older libuu-portal.so lacks uu_portal_set_cursor_mode.
UU_PORTAL_CURSOR_MODE=embedded sets the startup default.
* fix(portal): keep the cursor rebuild off locks and held input
Do not rebuild while the desktop is locked, and cancel an open that is already
waiting when it locks: GNOME closes the portal session on lock, so the request
would wait for consent that cannot be answered while holding `recovery`
against the post-unlock recovery.
Keys and buttons held through the portal input path are dropped when the
session is replaced, so wait for their release. Monitor selection takes the
same `recovery` lock, so it cannot install a stale source over the cursor
rebuild or the other way round, and the rebuilt session is revalidated against
the source it started from.
* fix(portal): carry the clipboard across cursor rebuilds
The engine keeps the viewer's text only in the session that provided it, and
the host re-sends it only for a new viewer update. A replacement session
therefore had no clipboard ownership, so changing the mouse mode made
previously synchronized text unpasteable until the viewer sent it again.
Offer the text to the replacement before the old session closes, but only
while that session still owns the selection: a local copy that has taken the
clipboard since is left alone. Touch consent upgrades replace the handle too,
so they take the same `recovery` lock as recovery, monitor selection and the
cursor rebuild and cannot install a handle opened under an older cursor mode.
* fix(portal): serialize clipboard writes with the cursor swap
The clipboard bridge calls clipboard_set from its own thread. Installing the
replacement handle and then carrying the old text over left a window where an
update could land on the new session and be overwritten by the older text,
with no pending update left to restore it.
Take a clipboard write lock before the handle is installed and hold it until
the carry-over finishes; clipboard_set takes the same lock. A write that
starts before the swap is included in the text carried over, a write that
starts after waits and wins.
* fix(portal): cancel a waiting rebuild when the desktop ends
The desktop ends its share when the last controller leaves (end_idle), but an
open that is still waiting for consent is not the installed handle, so it
survived that shutdown and installed a live share afterwards.
Bump an end generation in Session::end(), snapshot it before the open, and
cancel the request when it moves. The generation is checked again under the
handle lock before installing, so an end that races the swap either aborts
the rebuild or ends the handle it just installed.
* fix(portal): keep session swaps consistent with the viewer's state
Addresses the ownership race reported on the previous commit and the paths
that share its failure mode.
uu_portal_clipboard_set publishes the new text before SetSelection so a
transfer during the call serves it, restores the old text when the call fails,
and ignores a stale revision so a newer owner wins. The engine reports the
cursor mode it actually requested and the modes the backend advertised; the
host follows that instead of assuming the request was applied and stops
retrying where an embedded cursor is unavailable. The end generation is
sampled before the rebuild queues for `recovery`, a rebuild refuses to start
while the session is no longer alive, and cancellation now covers virtual
output opens.
Portal-bound keys and buttons are tracked separately, so an input held through
the portal defers the swap on every routing path, and the check is repeated
after the open. Every handle replacement (privacy screen, monitor switch,
touch consent, recovery after lock or stream loss) carries the viewer's
clipboard under the clipboard write lock, and the rebuilt session is checked
against the captured geometry.
* fix(portal): stop the swap bookkeeping from outliving a deferred rebuild
open_inner publishes the replacement's cursor mode before the swap. When held
input defers the rebuild, that session is never installed, so the host belief
described the discarded handle and later motion looked like it already matched.
Restore the belief from the installed session on every abort path.
The extra portal releases in release_held now stop at the first failure like
the sweep above; with a dead portal bus each remaining input waited out the
three-second timeout while the input and desktop-owner locks were held.
* fix(portal): invalidate clipboard ownership when a session ends
A closed session stops receiving owner changes, so a cached clipboard_owns
survived into the recovery paths and could republish old viewer text over a
newer local copy. Clear the claim in uu_portal_end and the Closed handler, and
let uu_portal_clipboard_copy refuse a source that is no longer alive.
Mark the embedded cursor unavailable when the engine did not apply it: with
AvailableCursorModes unreadable the session opens hidden, and retrying on every
motion run rebuilt it every 200 ms without changing the mode.
* fix(portal): retire held portal input when the devices are replaced
A key or button held through the native portal path is recorded when its
injection succeeds. If the session dies first, the key-up fails and the record
survives; recovery then installs new devices while the tracker still says an
input is held, so pointer-mode detection keeps deferring.
Keep the tracker in module state and clear it whenever the portal replaces or
ends its devices: the rebuild, both recovery paths and Session::end.
* fix(portal): scope held-input retirement to the retired handle
A press delivered to the replacement after the swap could be recorded while
the old handle was still being retired, and the unconditional reset then
deleted it. Hold the recording lock from before the swap until the reset
finishes, so only inputs sent to the retired devices are dropped.
A portal touchscreen contact was not tracked at all: the swap retired the
active contact while TouchDrag stayed down, so the following move reached a
session that never saw the press. Track contact down/up and include it in the
deferral state.
* fix(portal): serialize touch injection with handle replacement
inject_touch wrote the contact state outside the recording lock the handle
swap holds, so a contact delivered to the replacement between the swap and the
reset could be cleared, and the following phase-1 would fail on new devices
that never saw the press. Take the same lock across the injection and the
state update.
* fix(portal): decide the swap under the held-input lock
The final held-input check ran before install_replacement() took the serial
lock, so an input could be injected and recorded against the old handle while
the install waited for the clipboard lock. The install then retired the
records and swapped, destroying a contact the new devices never saw. Repeat
the check under the serial lock that recording uses, and defer the rebuild
when it hits.
* fix(portal): hold the input lock across native injection, refresh privacy geometry
inject_button and inject_key injected through the portal and only took the
recording lock afterwards, so a swap could take the lock, see no held input
and replace the devices that had just received the press. Take the serial lock
across the injection and its record, as the touch path does; the record
helpers no longer take it themselves.
A cursor rebuild that replaces the privacy virtual output invalidates the
cached pointer region: recompute it from the settled layout so absolute input
keeps landing inside the captured share.
The iOS viewer sends PrivateScreenSettingRequest{enable=0, privacy_type=3}
when the user turns the display off: its enable bit is the picture-mode
switch and the explicit type carries the intent. The host treated enable=0
as off, so the screen was restored instead of turned off.
Normalize that combination for iOS controllers only. Android 4.43.0 was
verified with the field-1 semantics, so keep Android and desktop viewers
unchanged. Record the platform after a desktop session is created, update
it on capture_desktop promotion, and cancel a pending UU_PRIVACY_AUTO even
when the switch is refused.
* input: park the pointer worker until pointer or gesture work exists
A fixed 4 ms tick kept ~250 wakeups/s (clock_gettime storms, worker
steal spins, pow-heavy flush math) burning ~3% CPU on an idle host with
no viewer connected. The worker now parks on a Notify permit until a
pointer/scroll/button event or a live touch gesture needs the 4 ms
cadence; notify_one permit semantics make the park check race-free.
Sub-notch scroll residue cannot inject alone, so it parks too.
* fix(input): preserve pointer wake ordering and motion cadence
Hold the serial lock before notifying button transitions so a parked worker cannot miss a newly pending gesture. Keep the 4 ms coalescing cadence until an empty tick and reset the interval after each parked phase.
Add regression coverage for button wake ordering, 1 ms motion streams, and idle cadence reset using Tokio test-only clock control.
---------
Co-authored-by: ferstar <[email protected]>
* feat(audio): capture and encode Opus in process
* feat(portal): encode VAAPI from leased DMA-BUF frames
* fix(input): route Portal controls and committed ASCII through uinput
* docs(portal): shorten setup notes and colocate hardware acceptance guidance
* fix(input): preserve GNOME IME guard for committed uinput text
* build: link pulse/opus after the static C archive (--as-needed)
Replayed as touch, a phone drag scrolled in browsers but still selected
text in VTE terminals (Ptyxis, GNOME Terminal), as on a real touchscreen.
By default on the portal backend the host now decides per press: moving
past 12 px within 450 ms scrolls smoothly (NotifyPointerAxis or EIS
scroll, content follows the finger, `finish` on release so clients may
coast), holding still selects with button 1 down, and a short press
clicks. This works in terminals too and needs no touchscreen consent.
UU_TOUCH_INPUT=touch keeps the touchscreen replay (with its one-time
consent upgrade); UU_TOUCH_INPUT=0 keeps plain mouse drags.
* portal: mobile touch-mode drags become touch, consent asks once for it
In touch mode the Android viewer sends a finger as a left-button drag at
absolute positions (observed: press, mouse_move_absolute stream, release),
which on GNOME selects text instead of scrolling. SelectDevices now also
asks for the touchscreen type where AvailableDeviceTypes offers it, and a
drag from an Android/iOS controller (controller_platform 2/3) that follows
absolute motion is replayed as one finger through NotifyTouch* or EIS
touch, so the desktop scrolls and taps as on a touch screen. Relative
motion (the trackpad mode) and every other viewer keep mouse buttons.
A consent saved before this restores keyboard and pointer only. The first
mobile drag then asks the host once per run in the background (GNOME's
share dialog, also visible on the phone; the consent timeout closes it)
and swaps the new session in only when it grants the touchscreen for the
same monitor and size. The saved consent is never removed beforehand.
UU_TOUCH_INPUT=0 keeps mouse drags and never asks.
* input: type mobile IME text as portal keysyms on Latin GNOME layouts
Each mobile text_input was set as the clipboard and pasted with
Shift+Insert, which VTE terminals bind to the PRIMARY selection: text
typed from the phone into a terminal pasted whatever was last selected
(observed: a selected line of box-drawing characters for every letter).
When GNOME's active input source (first of mru-sources) is a Latin xkb
layout, printable ASCII, newline and tab now go through
NotifyKeyboardKeysym, where Mutter picks the key and Shift level of the
active layout. Other text, IBus engines, KDE and EIS sessions keep the
clipboard paste, which no longer replaces the clipboard for plain typing.
* portal test: accept libei before 1.3 rejecting a touch outside one of two regions
libei 1.2 (Ubuntu 24.04, CI) accepts a touch only inside every region of the
device; 1.3.0 checks any region. The harness device also carries a region for
another screen, so the older library rejects the touch with a client-bug
diagnostic. A real one-stream session has one region.
* portal: switch monitors on GNOME, precise touch, stop sharing when idle
GNOME grants one monitor per portal consent and refuses multi-select, so
each Mutter monitor gets its own saved consent (gnome@<connector>). The
monitors are advertised as screens; a switch opens that monitor's session
(silently once consented, GNOME asks on the host the first time) and swaps
it into the shared PortalSession, whose frame size follows the monitor.
Absolute motion and clicks go through the portal session: a uinput tablet
spans every monitor, and clicks on the separate uinput queue could land
before the compositor moved the pointer.
An idle watch ends the portal session (screen-share indicator off) once no
desktop controller is left for two checks; the next one reopens it with the
saved consent.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* portal: stop screen sharing as soon as the last controller leaves
The idle watch stays as the fallback for controllers that vanish without
a release event.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* portal test: accept the runtime-engine message in portal builds
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>
* Wayland: type mobile text_input via clipboard paste on Portal and compositors without spare keycodes
* Wayland: address review comments for mobile text_input clipboard fallback
- Guard external clipboard commands in unit tests to eliminate desktop side effects
- Narrow paste fallback to Wayland and Portal backends only
- Document clipboard override and two-way sync behavior in README (both zh and en)
- Unify disable switch via UU_WAYLAND_TEXT=0
---------
Co-authored-by: shofukantkts <[email protected]>
Opt-in X11 monitor selection for #98: RandR monitors from xrandr --listmonitors through the existing screen-switch path; x11grab reads display+x,y at the monitor size, absolute input adds the monitor offset, in-process engine and xrandr resize are skipped while a region is captured.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
uu_wl_move returns -2 only when the compositor has no wlr-virtual-pointer; other failures are returned as before. Document UU_WAYLAND_NO_VIRTUAL_KEYBOARD.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Compositors without wlr-virtual-pointer, or that restrict the virtual keyboard to their input method (Denial), now bring up capture and fall back to uinput for input. UU_WAYLAND_NO_VIRTUAL_KEYBOARD=1 skips the virtual keyboard.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The mobile clients' input method sends `text_input {content}` over CONTROL. Upstream only
implements it for X11, so a Wayland host (Hyprland here) dropped the characters with a
warning. Wire it up: characters the session keymap already produces are pressed as
ordinary keycodes, and characters it cannot produce are carried by spare keycodes (the
keymap has ~33 named but unmapped ones) declared for the duration of one batch, in text
order, then removed again.
The focused client translates keycodes with the keymap the compositor last handed it, and
a keymap uploaded to a keyboard the seat already has is never handed on again: measured on
Hyprland 0.56.2, pressing the remapped keycodes then reproduces the previous message's
text (a space came out as the previous message's first character, and every message
repeated the one before it). Re-attaching the virtual keyboard is what makes the
compositor send the keymap to the client, so each batch does that, waits for the
compositor to install the keymap, and releases held keys first because the new keyboard
starts with none. No timing sleep is needed once the keyboard is re-attached, so there is
none. Plain latin text and spaces never touch the keymap at all. The pristine keymap is
restored after every call, so uu_wl_key keeps its own symbol map and the compositor is
never left with spare keys bound.
`wayland-text-probe` reports how many characters one upload can carry and types a sample;
UU_PROBE_SEQUENCE types one message per |-separated field, the way the phone's input
method sends them, which is what reproduced this locally.
Co-authored-by: Hey Claw <[email protected]>
Co-authored-by: Claude Code <[email protected]>
Host: accept the official text_input {content} action on any channel and type it on
X11 via spare-keycode Unicode keysyms (verified with xev); log non-CONTROL JSON
action names. Viewer: Windows-type hosts get only the case-10 text change (a rich
format list overwrote it and was never rendered), Mac gets only the rich offer.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Log unsupported CONTROL input actions (names only) and the server NumLock
state for numpad keys; skip the SDL window test on macOS (main-thread SDL).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* fix(portal): establish locked desktop control before capture
* test(portal): preserve v4 pixel inventory after deferred capture settings
* fix(control): refresh lock state after viewer Echo handshake
* fix(portal): announce desktop target before locked capture wait
* ci: run the desktop_power native test under ASan/UBSan
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
* input: skip the Portal release sweep while no session is bound
The first locked connection creates its peer before any Portal exists; the sweep
then logged two failed injections and a 'reset incomplete' warning for nothing.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
* docs: describe locked cold connections, auto unlock limits and iPhone status
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
* fix(portal): treat a cached screen with a closed Portal session as not ready
begin() now marks the state as waiting when the cached session is dead, and
permission/video/clipboard consumers honor it instead of reporting Allow and
starting the stale video path before maintain() has rebuilt capture.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
* fix(portal): gate input on a ready desktop and share recovery backoff
Input dispatch kept only the cached screen and dropped State.waiting, so a closed
Portal passed the gate once the screen unlocked. The supervisor's recover() and the
encoder's stream-loss recovery now share failure backoff and a terminal-denial flag,
so a denied consent is not re-prompted by the other path.
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
* fix(control): keep unknown lock state unlocked, stop waits on Closing, never drop capture settings
- QuerySystemState with no logind lock state (Xvfb/linger) reports unlocked as before.
- The Portal wait loop also exits while the channel is Closing.
- Capture-setting requests back-pressure instead of try_send dropping the latest setting.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---------
Co-authored-by: Claude Sonnet 5.5 <[email protected]>
Co-authored-by: VallierDev <[email protected]>
With the viewer option 「使用 macOS 快捷键」 off, Cmd arrives as VK_LWIN
(verified: Cmd 0x5B, Option 0xA4, Ctrl 0xA2; controller_platform 4 = macOS).
For macOS controllers Cmd+C/V/A/F/X/Z become Ctrl+key, Cmd+C/V become
Ctrl+Shift+C/V in a focused terminal, Ctrl+C keeps interrupting; other
combos and Cmd alone stay Super. Verified with the official Mac viewer.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Squashed from the Codex v0.1.9 branch (reviewed):
- Clear stale input when a desktop session starts or ends and at service
start/stop: X11 releases every pressed key/button on the display (a stuck
XTest button blocked all clicks after a VM reboot), uinput resets its own
devices, Wayland/portal sweep the keys this host can emit. A per-desktop
lease stops late callbacks of an old connection from touching a new one;
file/terminal/port-mapping peers never release desktop input.
- Native Wayland (--features wayland): double-buffered screencopy pipeline,
viewer frame rate drives the capture rate, opt-in dma-buf (UU_WAYLAND_DMABUF=1,
linear only, falls back to SHM). VM-measured only.
- Internal QA records (docs/validation, contrib/tests) removed from the public
tree and ignored; they stay in the private project.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- uinput.rs: kernel virtual keyboard + absolute pointer (Sunshine-style),
physical evdev keycodes so the compositor's keysym lookup cannot drop
shifted symbols (#3). Auto on Wayland/portal when /dev/uinput is
accessible (udev uaccess rule), XTest kept on X11 (it suppresses
autorepeat for held keys over latency). UU_INPUT_BACKEND=auto|uinput|native.
- Mac viewers send Cmd+V as Ctrl+V (verified: Cmd and Ctrl both arrive as
VK_LCONTROL). In a focused X11 terminal Ctrl+V becomes Ctrl+Shift+V so it
pastes instead of quoting the next key; Ctrl+C always interrupts (copy is
Cmd+Shift+C). UU_TERMINAL_SHORTCUTS=0 disables.
Verified on the 3090 with the official Mac viewer: paste, interrupt,
uinput typing incl. underscores.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Unofficial Linux host for NetEase UU Remote: X11 (production-tested on
GNOME/Ubuntu 24.04), experimental Wayland (Hyprland) and portal
(GNOME/KDE) backends; video, input, clipboard, file transfer, terminal,
sound, quality/fps/resolution settings, lock/unlock and system actions.
Co-Authored-By: Claude Opus 5.5 <[email protected]>