122 lines
6.5 KiB
Markdown
122 lines
6.5 KiB
Markdown
# Capture frame protocol
|
||
|
||
How frames travel from the compositor (via PipeWire) into the client's
|
||
capture calls. Multiple selected monitor streams are composed into one virtual
|
||
desktop frame in the shared transport; the Windows hook crops each monitor
|
||
request from that frame.
|
||
|
||
```text
|
||
xdg-desktop-portal ScreenCast
|
||
│ (portal-consented session; one stream per selected monitor)
|
||
▼
|
||
uur-pw-capture (C) ── PipeWire streams, BGRx, desktop composition
|
||
│ writes slots, seqlock per slot
|
||
▼
|
||
$XDG_RUNTIME_DIR/uur/frames.v3 (mode 0600, 704-byte header + 3 slots)
|
||
│ mapped read-only by the PE hook through its configured Wine Z: path
|
||
▼
|
||
uur-hook.dll ── intercepts static/dynamic gdi32!BitBlt/StretchBlt calls;
|
||
scales the newest stable snapshot into the destination DIB
|
||
```
|
||
|
||
## Shared file layout
|
||
|
||
All integers little-endian. Total size = 704 + 3 × (8 + stride × height).
|
||
|
||
| offset | type | field |
|
||
| --- | --- | --- |
|
||
| 0 | u32 | magic `0x46525555` ("UURF") |
|
||
| 4 | u32 | protocol version = 3 |
|
||
| 8 | u32 | frame width (px) |
|
||
| 12 | u32 | frame height (px) |
|
||
| 16 | u32 | stride (bytes per row) |
|
||
| 20 | u32 | format: 1 = BGRx (32 bits per pixel) |
|
||
| 24 | u64 | total frames written (monotonic counter) |
|
||
| 32 | u32 | flags: bit 0 means the frame is a virtual-desktop composite |
|
||
| 36 | u32 | monitor layout record count (1..16) |
|
||
| 40 | u64 | producer CLOCK_MONOTONIC heartbeat in milliseconds; zero if unsupported |
|
||
| 48..64 | — | reserved, zero |
|
||
|
||
The fixed-size monitor table starts at offset 64 and contains 16 records of
|
||
40 bytes each. Unused records are zero. Each record contains signed logical
|
||
`x` and `y`, logical `width` and `height`, then unsigned `frame_x`, `frame_y`,
|
||
`frame_width` and `frame_height` describing that monitor's rectangle in the
|
||
composited frame, then `capture_width` and `capture_height` for the negotiated
|
||
source stream resolution. A single-stream capture has one record covering the
|
||
full frame.
|
||
|
||
Slot *i* (i = 0, 1, 2) at offset `704 + i × (8 + stride × height)`:
|
||
|
||
| offset | type | field |
|
||
| --- | --- | --- |
|
||
| 0 | u64 | sequence: `2×n − 1` while slot n is being written, `2×n` when slot n is stable |
|
||
| 8 | bytes | frame, `stride × height`, rows top-to-bottom, pixels BGRx |
|
||
|
||
Writer picks slots round-robin (`n mod 3`), bumps the counter, publishes
|
||
the odd sequence (release), copies the frame, then publishes the even
|
||
sequence (release). The reader is a classic seqlock read: load the
|
||
sequence, copy, re-load; retry while the value changed or is odd. No
|
||
mutexes; a torn read is detected, never observed.
|
||
|
||
The transport path is created per user and passed to the Wine adapter through
|
||
the managed prefix configuration. It is never a global shared-memory name.
|
||
|
||
## Geometry and session changes
|
||
|
||
For one selected stream, an exact Wine monitor rectangle maps onto the entire
|
||
frame when their aspect ratios match. With multiple selected monitor streams,
|
||
the producer uses portal position/size metadata to compose the streams into a
|
||
virtual desktop frame; Wine monitor rectangles then crop their matching region.
|
||
If a portal omits layout metadata, streams are arranged left to right. Fractional
|
||
scaling is normalized to a shared frame scale; each stream is resampled into
|
||
its reported logical rectangle. The hook matches Wine monitor bounds to the
|
||
captured layout and crops in monitor-local coordinates, so differing monitor
|
||
resolutions and vertical offsets do not shift the top edge of a screen.
|
||
The capture service's virtual display API uses the composed frame rectangles
|
||
relative to the primary monitor, including a common pixel scale. UI and WebView
|
||
processes retain native display geometry. Raw source pixel sizes remain metadata
|
||
and are never combined with logical origins. Portals without stable output
|
||
identity require the selected monitor count to match Wine's monitor count for
|
||
per-monitor matching; otherwise the composite is used as a whole.
|
||
|
||
Multiple monitors are explicitly enabled with `uur display source monitor
|
||
--multiple`, followed by a new session and monitor selection in the portal.
|
||
Upgrading preserves existing grants. Choosing `uur display source monitor`
|
||
returns to a single selected source. Absolute input selects the consented node
|
||
containing the logical position, then sends that node's local coordinates.
|
||
|
||
Each transport inode has immutable geometry. A negotiated size change creates
|
||
a new mode-0600 file, initializes its header and slots, then atomically replaces
|
||
the transport path. The previous mapping remains valid for existing readers;
|
||
the producer never truncates it. The Wine adapter checks file identity at least
|
||
every 250 ms and remaps replacements while serializing its capture readers.
|
||
|
||
The PipeWire consumer accepts explicitly linear DMA-BUFs and retains a separate
|
||
shared-memory format offer. DMA-BUF reads use the source stride, plane offsets,
|
||
fd extent, readiness fence and CPU-access synchronization. Implicit modifiers
|
||
require a graphics importer and are not advertised by this CPU consumer.
|
||
|
||
A stream error, lost connection or ten seconds in PAUSED after streaming makes
|
||
the helper exit with status 75. A ten-second expired producer heartbeat also
|
||
restarts a hung helper, independently of desktop changes. Both ScreenCast and
|
||
combined RemoteDesktop supervisors retry up to three
|
||
times using the saved restore token, with a one-second backoff. Thirty seconds
|
||
of healthy capture resets the retry budget. An unchanged desktop in STREAMING
|
||
does not trigger this watchdog. Recovery keeps the saved token and performs no
|
||
token-less fallback or switch to direct capture. The portal controls whether
|
||
restoring that choice requires confirmation. A portal Session.Closed signal
|
||
without a matching Mutter layout change ends sharing; the user starts a new
|
||
session explicitly. The supervisor subscribes to the raw signal on the session's
|
||
own connection, accepting both dictionary and empty signal bodies.
|
||
|
||
When Mutter's DisplayConfig service is already present, the supervisor also
|
||
subscribes to MonitorsChanged and proactively restores capture after a layout
|
||
change. It coalesces signals during the reconnect delay. Mutter may close the
|
||
old session before forwarding MonitorsChanged, so a Closed signal has a 250 ms
|
||
window to correlate that transaction. Closed without a matching change stops
|
||
capture, and process stop requests always take priority. This optional listener
|
||
never starts a GNOME service on other desktops.
|
||
Successful layout-driven restarts do not consume the failure retry budget.
|
||
Only the initial session can make a token-less interactive selection; later
|
||
restores keep that boundary even when the failure counter has been reset.
|