feat(capture): support consented multi-monitor desktops with recovery (#32)
This commit is contained in:
1 parent
6d0e733cd7
commit
98af6fce28
19 files changed
+2175
-271
No files matched your search
@@ -62,7 +62,7 @@ RAII cleanup terminates only those helper groups and the one managed Wine
|
||||
prefix. There are no permanent daemons, global `pkill` calls, or required
|
||||
systemd units.
|
||||
|
||||
Frame data lives in `$XDG_RUNTIME_DIR/uur/frames.v1` with mode 0600. The path is
|
||||
Frame data lives in `$XDG_RUNTIME_DIR/uur/frames.v3` with mode 0600. The path is
|
||||
passed through the adapter configuration, so users and concurrent sessions do
|
||||
not share a global `/dev/shm` object.
|
||||
|
||||
|
||||
+45
-15
@@ -1,16 +1,18 @@
|
||||
# Capture frame protocol
|
||||
|
||||
How frames travel from the compositor (via PipeWire) into the client's
|
||||
capture calls. Two producers/consumers, one shared file, no locks.
|
||||
capture calls. Multiple selected monitor streams are composed into one virtual
|
||||
desktop frame in the shared transport; the Windows hook crops each monitor
|
||||
request from that frame.
|
||||
|
||||
```text
|
||||
xdg-desktop-portal ScreenCast
|
||||
│ (portal-consented session; per-monitor)
|
||||
│ (portal-consented session; one stream per selected monitor)
|
||||
▼
|
||||
uur-pw-capture (C) ── PipeWire stream, BGRx, SHM buffers
|
||||
uur-pw-capture (C) ── PipeWire streams, BGRx, desktop composition
|
||||
│ writes slots, seqlock per slot
|
||||
▼
|
||||
$XDG_RUNTIME_DIR/uur/frames.v1 (mode 0600, 64-byte header + 3 slots)
|
||||
$XDG_RUNTIME_DIR/uur/frames.v3 (mode 0600, 704-byte header + 3 slots)
|
||||
│ mapped read-only by the PE hook through its configured Wine Z: path
|
||||
▼
|
||||
uur-hook.dll ── intercepts static/dynamic gdi32!BitBlt/StretchBlt calls;
|
||||
@@ -19,20 +21,31 @@ uur-hook.dll ── intercepts static/dynamic gdi32!BitBlt/StretchBlt calls;
|
||||
|
||||
## Shared file layout
|
||||
|
||||
All integers little-endian. Total size = 64 + 3 × (8 + stride × height).
|
||||
All integers little-endian. Total size = 704 + 3 × (8 + stride × height).
|
||||
|
||||
| offset | type | field |
|
||||
| --- | --- | --- |
|
||||
| 0 | u32 | magic `0x46525555` ("UURF") |
|
||||
| 4 | u32 | protocol version = 1 |
|
||||
| 4 | u32 | protocol version = 3 |
|
||||
| 8 | u32 | frame width (px) |
|
||||
| 12 | u32 | frame height (px) |
|
||||
| 16 | u32 | stride (bytes per row) |
|
||||
| 20 | u32 | format: 1 = BGRx (32 bits per pixel) |
|
||||
| 24 | u64 | total frames written (monotonic counter) |
|
||||
| 32..64 | — | reserved, zero |
|
||||
| 32 | u32 | flags: bit 0 means the frame is a virtual-desktop composite |
|
||||
| 36 | u32 | monitor layout record count (1..16) |
|
||||
| 40 | u64 | producer CLOCK_MONOTONIC heartbeat in milliseconds; zero if unsupported |
|
||||
| 48..64 | — | reserved, zero |
|
||||
|
||||
Slot *i* (i = 0, 1, 2) at offset `64 + i × (8 + stride × height)`:
|
||||
The fixed-size monitor table starts at offset 64 and contains 16 records of
|
||||
40 bytes each. Unused records are zero. Each record contains signed logical
|
||||
`x` and `y`, logical `width` and `height`, then unsigned `frame_x`, `frame_y`,
|
||||
`frame_width` and `frame_height` describing that monitor's rectangle in the
|
||||
composited frame, then `capture_width` and `capture_height` for the negotiated
|
||||
source stream resolution. A single-stream capture has one record covering the
|
||||
full frame.
|
||||
|
||||
Slot *i* (i = 0, 1, 2) at offset `704 + i × (8 + stride × height)`:
|
||||
|
||||
| offset | type | field |
|
||||
| --- | --- | --- |
|
||||
@@ -50,12 +63,27 @@ the managed prefix configuration. It is never a global shared-memory name.
|
||||
|
||||
## Geometry and session changes
|
||||
|
||||
An exact Wine monitor rectangle maps onto the entire selected frame when their
|
||||
aspect ratios match. Pixel dimensions may differ under fractional scaling, where
|
||||
Xwayland's render size can exceed the portal's native frame size. Partial regions
|
||||
and whole-virtual-desktop requests retain the existing virtual-desktop mapping.
|
||||
This does not establish monitor identity between Wine and the portal; selecting
|
||||
a non-primary monitor still requires consistent client/input coordinates.
|
||||
For one selected stream, an exact Wine monitor rectangle maps onto the entire
|
||||
frame when their aspect ratios match. With multiple selected monitor streams,
|
||||
the producer uses portal position/size metadata to compose the streams into a
|
||||
virtual desktop frame; Wine monitor rectangles then crop their matching region.
|
||||
If a portal omits layout metadata, streams are arranged left to right. Fractional
|
||||
scaling is normalized to a shared frame scale; each stream is resampled into
|
||||
its reported logical rectangle. The hook matches Wine monitor bounds to the
|
||||
captured layout and crops in monitor-local coordinates, so differing monitor
|
||||
resolutions and vertical offsets do not shift the top edge of a screen.
|
||||
The capture service's virtual display API uses the composed frame rectangles
|
||||
relative to the primary monitor, including a common pixel scale. UI and WebView
|
||||
processes retain native display geometry. Raw source pixel sizes remain metadata
|
||||
and are never combined with logical origins. Portals without stable output
|
||||
identity require the selected monitor count to match Wine's monitor count for
|
||||
per-monitor matching; otherwise the composite is used as a whole.
|
||||
|
||||
Multiple monitors are explicitly enabled with `uur display source monitor
|
||||
--multiple`, followed by a new session and monitor selection in the portal.
|
||||
Upgrading preserves existing grants. Choosing `uur display source monitor`
|
||||
returns to a single selected source. Absolute input selects the consented node
|
||||
containing the logical position, then sends that node's local coordinates.
|
||||
|
||||
Each transport inode has immutable geometry. A negotiated size change creates
|
||||
a new mode-0600 file, initializes its header and slots, then atomically replaces
|
||||
@@ -69,7 +97,9 @@ fd extent, readiness fence and CPU-access synchronization. Implicit modifiers
|
||||
require a graphics importer and are not advertised by this CPU consumer.
|
||||
|
||||
A stream error, lost connection or ten seconds in PAUSED after streaming makes
|
||||
the helper exit with status 75. The ScreenCast supervisor retries up to three
|
||||
the helper exit with status 75. A ten-second expired producer heartbeat also
|
||||
restarts a hung helper, independently of desktop changes. Both ScreenCast and
|
||||
combined RemoteDesktop supervisors retry up to three
|
||||
times using the saved restore token, with a one-second backoff. Thirty seconds
|
||||
of healthy capture resets the retry budget. An unchanged desktop in STREAMING
|
||||
does not trigger this watchdog. Recovery keeps the saved token and performs no
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
| X11 multi-monitor | Wine integration | Native Wine/XRandR display topology |
|
||||
| Wayland monitor/window selection | available | Persistent portal source selection |
|
||||
| Wayland virtual source | available when advertised | `uur display source virtual` → ScreenCast Virtual source |
|
||||
| Independent Wayland multi-screen switching | in progress | Multiple PipeWire streams → UU display topology adapter |
|
||||
| Wayland multi-screen capture | available | Multiple PipeWire streams → virtual desktop composite → per-monitor crops |
|
||||
| Clipboard and phone IME text | planned | RemoteDesktop Clipboard portal, then X11 fallback |
|
||||
| Speaker and microphone audio | available through Wine | PulseAudio, pipewire-pulse, or ALSA selected from live capabilities |
|
||||
| System-audio loopback and remote mute | Wine integration | WASAPI → winepulse/winealsa |
|
||||
|
||||
@@ -210,14 +210,17 @@ choice; this is independent from PipeWire's Wayland video role.
|
||||
```bash
|
||||
uur display status
|
||||
uur display source monitor
|
||||
uur display source monitor --multiple
|
||||
uur display source window
|
||||
uur display source virtual
|
||||
```
|
||||
|
||||
The command clears the old restore token so the next Portal session can create
|
||||
or select the requested source. X11 exposes its XRandR topology to Wine.
|
||||
Wayland currently carries one selected stream; independent multi-stream
|
||||
switching is the next display-topology adapter.
|
||||
or select the requested source. With `monitor --multiple`, the chooser can select
|
||||
multiple displays; uur composes those PipeWire streams into the virtual desktop
|
||||
frame and crops each Wine monitor request from it. X11 exposes its XRandR
|
||||
topology to Wine directly. Upgrades preserve existing restore tokens; multiple
|
||||
monitor selection is an explicit choice and applies to the next session.
|
||||
|
||||
## First run and normal use
|
||||
|
||||
|
||||
@@ -3,11 +3,12 @@
|
||||
## Frame sources
|
||||
|
||||
- X11 sessions expose the XRandR monitor topology directly to Wine.
|
||||
- Wayland monitor and window sources use ScreenCast/PipeWire.
|
||||
- Wayland monitor and window sources use ScreenCast/PipeWire. Monitor selection
|
||||
supports multiple outputs with `uur display source monitor --multiple` and
|
||||
composes their streams into the virtual desktop
|
||||
frame consumed by the Wine capture adapter.
|
||||
- `uur display source virtual` requests the portal's Virtual source type and
|
||||
stores its restore token.
|
||||
- A multi-stream compositor will combine or expose several PipeWire nodes to
|
||||
UU's display topology for independent Wayland screen switching.
|
||||
|
||||
## Codec capabilities
|
||||
|
||||
|
||||
Reference in new issue
Block a user