Files
uur/docs/capture-protocol.md
T

122 lines
6.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Capture frame protocol
How frames travel from the compositor (via PipeWire) into the client's
capture calls. Multiple selected monitor streams are composed into one virtual
desktop frame in the shared transport; the Windows hook crops each monitor
request from that frame.
```text
xdg-desktop-portal ScreenCast
│ (portal-consented session; one stream per selected monitor)
▼
uur-pw-capture (C) ── PipeWire streams, BGRx, desktop composition
│ writes slots, seqlock per slot
▼
$XDG_RUNTIME_DIR/uur/frames.v3 (mode 0600, 704-byte header + 3 slots)
│ mapped read-only by the PE hook through its configured Wine Z: path
▼
uur-hook.dll ── intercepts static/dynamic gdi32!BitBlt/StretchBlt calls;
scales the newest stable snapshot into the destination DIB
```
## Shared file layout
All integers little-endian. Total size = 704 + 3 × (8 + stride × height).
| offset | type | field |
| --- | --- | --- |
| 0 | u32 | magic `0x46525555` ("UURF") |
| 4 | u32 | protocol version = 3 |
| 8 | u32 | frame width (px) |
| 12 | u32 | frame height (px) |
| 16 | u32 | stride (bytes per row) |
| 20 | u32 | format: 1 = BGRx (32 bits per pixel) |
| 24 | u64 | total frames written (monotonic counter) |
| 32 | u32 | flags: bit 0 means the frame is a virtual-desktop composite |
| 36 | u32 | monitor layout record count (1..16) |
| 40 | u64 | producer CLOCK_MONOTONIC heartbeat in milliseconds; zero if unsupported |
| 48..64 | — | reserved, zero |
The fixed-size monitor table starts at offset 64 and contains 16 records of
40 bytes each. Unused records are zero. Each record contains signed logical
`x` and `y`, logical `width` and `height`, then unsigned `frame_x`, `frame_y`,
`frame_width` and `frame_height` describing that monitor's rectangle in the
composited frame, then `capture_width` and `capture_height` for the negotiated
source stream resolution. A single-stream capture has one record covering the
full frame.
Slot *i* (i = 0, 1, 2) at offset `704 + i × (8 + stride × height)`:
| offset | type | field |
| --- | --- | --- |
| 0 | u64 | sequence: `2×n − 1` while slot n is being written, `2×n` when slot n is stable |
| 8 | bytes | frame, `stride × height`, rows top-to-bottom, pixels BGRx |
Writer picks slots round-robin (`n mod 3`), bumps the counter, publishes
the odd sequence (release), copies the frame, then publishes the even
sequence (release). The reader is a classic seqlock read: load the
sequence, copy, re-load; retry while the value changed or is odd. No
mutexes; a torn read is detected, never observed.
The transport path is created per user and passed to the Wine adapter through
the managed prefix configuration. It is never a global shared-memory name.
## Geometry and session changes
For one selected stream, an exact Wine monitor rectangle maps onto the entire
frame when their aspect ratios match. With multiple selected monitor streams,
the producer uses portal position/size metadata to compose the streams into a
virtual desktop frame; Wine monitor rectangles then crop their matching region.
If a portal omits layout metadata, streams are arranged left to right. Fractional
scaling is normalized to a shared frame scale; each stream is resampled into
its reported logical rectangle. The hook matches Wine monitor bounds to the
captured layout and crops in monitor-local coordinates, so differing monitor
resolutions and vertical offsets do not shift the top edge of a screen.
The capture service's virtual display API uses the composed frame rectangles
relative to the primary monitor, including a common pixel scale. UI and WebView
processes retain native display geometry. Raw source pixel sizes remain metadata
and are never combined with logical origins. Portals without stable output
identity require the selected monitor count to match Wine's monitor count for
per-monitor matching; otherwise the composite is used as a whole.
Multiple monitors are explicitly enabled with `uur display source monitor
--multiple`, followed by a new session and monitor selection in the portal.
Upgrading preserves existing grants. Choosing `uur display source monitor`
returns to a single selected source. Absolute input selects the consented node
containing the logical position, then sends that node's local coordinates.
Each transport inode has immutable geometry. A negotiated size change creates
a new mode-0600 file, initializes its header and slots, then atomically replaces
the transport path. The previous mapping remains valid for existing readers;
the producer never truncates it. The Wine adapter checks file identity at least
every 250 ms and remaps replacements while serializing its capture readers.
The PipeWire consumer accepts explicitly linear DMA-BUFs and retains a separate
shared-memory format offer. DMA-BUF reads use the source stride, plane offsets,
fd extent, readiness fence and CPU-access synchronization. Implicit modifiers
require a graphics importer and are not advertised by this CPU consumer.
A stream error, lost connection or ten seconds in PAUSED after streaming makes
the helper exit with status 75. A ten-second expired producer heartbeat also
restarts a hung helper, independently of desktop changes. Both ScreenCast and
combined RemoteDesktop supervisors retry up to three
times using the saved restore token, with a one-second backoff. Thirty seconds
of healthy capture resets the retry budget. An unchanged desktop in STREAMING
does not trigger this watchdog. Recovery keeps the saved token and performs no
token-less fallback or switch to direct capture. The portal controls whether
restoring that choice requires confirmation. A portal Session.Closed signal
without a matching Mutter layout change ends sharing; the user starts a new
session explicitly. The supervisor subscribes to the raw signal on the session's
own connection, accepting both dictionary and empty signal bodies.
When Mutter's DisplayConfig service is already present, the supervisor also
subscribes to MonitorsChanged and proactively restores capture after a layout
change. It coalesces signals during the reconnect delay. Mutter may close the
old session before forwarding MonitorsChanged, so a Closed signal has a 250 ms
window to correlate that transaction. Closed without a matching change stops
capture, and process stop requests always take priority. This optional listener
never starts a GNOME service on other desktops.
Successful layout-driven restarts do not consume the failure retry budget.
Only the initial session can make a token-less interactive selection; later
restores keep that boundary even when the failure counter has been reset.