petere
2b85a78c22
- Remove the project-local gitea MCP (opencode.json) and gitea-mcp-server from the dev shell — the gitea MCP is now installed globally and points at https://gitea.edley.me - Update README/project.md links and the flake package homepage to the new public instance - The GITEA_ACCESS_TOKEN for the old homeserver is removed from .secrets.env (gitignored); the global token from /run/secrets is used
172 lines
10 KiB
Markdown
172 lines
10 KiB
Markdown
# TeleportFling - Project Overview
|
|
|
|
## Summary
|
|
|
|
TeleportFling is a standalone Linux application that captures a single screen along with
|
|
audio from the host machine and transmits the combined stream over a LAN using the
|
|
[Teleport protocol](https://github.com/fzwoch/obs-teleport). The teleport stream is received
|
|
by another machine running [OBS Studio](https://obsproject.com) with the `obs-teleport`
|
|
plugin installed, where it appears as a regular Teleport source.
|
|
|
|
The project is an OBS/NDI-like replacement for a minimal multi-machine streaming setup:
|
|
one machine (the "flinger") produces a screen + audio feed, a second machine aggregates
|
|
and streams it. No NDI compatibility of any form.
|
|
|
|
## Goals
|
|
|
|
1. Capture a single monitor (screen) + configurable audio source on Linux.
|
|
2. Push the stream over the LAN using the Teleport protocol so an OBS + `obs-teleport`
|
|
receiver can consume it with zero plugin-side configuration on the source machine.
|
|
3. Ship a standalone, installable application, not an OBS plugin:
|
|
- Full configuration GUI (device, audio source, quality, discovery settings).
|
|
- System tray icon for quick start/stop and status.
|
|
- Works headless-ish: capture/transmit logic is decoupled from the GUI so it can
|
|
later run as a daemon or be scripted.
|
|
|
|
## Non-Goals
|
|
|
|
- NDI or DistroAV compatibility.
|
|
- Software transcoding beyond the JPEG frame encoding the protocol requires.
|
|
- Synchronous multi-screen capture initially (single screen only, per the name).
|
|
- Windows/macOS support initially (Linux only).
|
|
|
|
## Target Environment
|
|
|
|
| Concern | Decision |
|
|
|--------------------|----------------------------------------------------------------|
|
|
| Language | Go (same language as `obs-teleport`) |
|
|
| Display server | Wayland primary; X11 support if it falls out cheaply |
|
|
| Audio | Configurable: system/desktop audio (default) or mic/input device |
|
|
| Screen capture | Wayland: PipeWire / xdg-desktop-portal; X11: X11 extension or PipeWire fallback |
|
|
| GUI | Fyne (config window + system tray) |
|
|
| Discovery | UDP multicast peer discovery (identical to `obs-teleport`) |
|
|
| Target quality | Initial release: 1920x1080 @ 30 fps, balanced JPEG quality |
|
|
| Protocol target | Latest `obs-teleport` release (protocol contract below) |
|
|
| Remote git | Public self-hosted Gitea at `https://gitea.edley.me/` |
|
|
|
|
## Teleport Protocol Contract
|
|
|
|
Reference implementation: https://github.com/fzwoch/obs-teleport (GPL-2.0).
|
|
|
|
1. **Peer discovery** - UDP multicast broadcast of a JSON `AnnouncePayload`:
|
|
`{"Name", "Port", "AudioAndVideo", "Version", "Address"}`. The OBS receiver uses
|
|
these announcements to populate its source list. `obs-teleport` uses
|
|
`github.com/schollz/peerdiscovery`.
|
|
2. **Transport** - TCP. The sender binds a listener on a configurable port and accepts
|
|
connections from receivers (multiple concurrent receivers supported).
|
|
3. **Packet header** (all little-endian):
|
|
- `Header`: `Type [4]byte`, `Timestamp uint64`, `Size int32`.
|
|
- `ImageHeader` (video only): `ColorMatrix [16]float32`, `ColorRangeMin [3]float32`,
|
|
`ColorRangeMax [3]float32` (from OBS video format color parameters).
|
|
- `WaveHeader` (audio only): `Format int32`, `SampleRate int32`, `Speakers int32`,
|
|
`Frames int32`.
|
|
4. **Video frames** - `Type = "JPEG"`. Frames are compressed to JPEG (turbojpeg) with
|
|
configurable quality. Payload = header + image header + JPEG bytes.
|
|
5. **Audio** - `Type = "WAVE"`. Raw interleaved PCM. Payload = header + wave header +
|
|
PCM samples (16-bit stereo @ 48 kHz typical).
|
|
6. **Ordering** - Video frames are queued, encoded in order, and sent in the same order;
|
|
dropped frames are counted as lagged frames. Audio is sent immediately as it arrives.
|
|
7. **Compatibility contract** - We must remain byte-compatible with the latest
|
|
`obs-teleport` release so that stock OBS Teleport receivers can discover and decode
|
|
our stream without modification.
|
|
|
|
## Architecture (proposed)
|
|
|
|
```
|
|
┌────────────────────────────┐ ┌───────────────────────────────┐
|
|
│ Fling app │ │ OBS receiver │
|
|
│ ┌──────────┐ ┌─────────┐ │ │ │
|
|
│ │ Screen │ │ Audio │ │ │ ┌─────────────────────────┐ │
|
|
│ │ capture │ │ capture │ │ │ │ obs-teleport source │ │
|
|
│ └────┬─────┘ └────┬────┘ │ │ └───────────┬─────────────┘ │
|
|
│ │ │ │ │ │ │
|
|
│ ▼ ▼ │ │ ▼ │
|
|
│ ┌────────────────────────┐ │ │ ┌─────────────────────────┐ │
|
|
│ │ Flinger core │ │ │ │ (decode + render) │ │
|
|
│ │ · JPEG encode (video) │ ├─────┼─▶│ OBS Studio pipeline │ │
|
|
│ │ · interleave (audio) │ │ │ └─────────────────────────┘ │
|
|
│ │ · packetize │ │ │ │
|
|
│ │ · TCP sender │ │ │ │
|
|
│ │ · UDP announcer │ │ │ │
|
|
│ └───────────┬────────────┘ │ │ │
|
|
└──────────────┼──────────────┘ └───────────────────────────────┘
|
|
│ LAN (TCP + UDP multicast)
|
|
```
|
|
|
|
Components:
|
|
|
|
1. **Core library (`internal/flinger`)** - protocol + capture agnostic:
|
|
- `protocol`: packet types, JPEG encode, WAVE packetize, header (de)serialization.
|
|
- `output`: TCP listener, connection management (reuse `obs-teleport`'s per-connection
|
|
buffered channel approach with drop-when-overflow), framing/ordering queue.
|
|
- `discovery`: UDP multicast announcer (`AnnouncePayload`).
|
|
- `capture`: interface `CaptureSource` + implementations for screen and audio.
|
|
2. **GUI (`cmd/teleportfling` or `internal/gui`)** - Fyne:
|
|
- Settings window: monitor picker, audio source picker, port, JPEG quality,
|
|
resolution/fps targets, discovery on/off, enable/disable.
|
|
- System tray: start/stop toggle, status indicator, open settings, quit.
|
|
- Config persistence (JSON config file, e.g. `~/.config/teleportfling/config.json`).
|
|
3. **Capture backend**:
|
|
- Wayland: PipeWire via `pipewire` Go bindings or `xdg-desktop-portal` screen capture.
|
|
- Audio: PipeWire/PulseAudio monitor source (system audio) or input device; configurable.
|
|
- X11 (optional): X11 API screen grab or PipeWire fallback where available.
|
|
|
|
## Milestones
|
|
|
|
1. **M1 - Protocol proof of life**: static JPEG frames + synthesized audio sent over TCP,
|
|
multicast announcements read successfully by OBS Teleport receiver. CLI only.
|
|
2. **M2 - Real capture**: PipeWire screen capture + system audio capture feeding the
|
|
core; confirm 1080p30 on Wayland.
|
|
3. **M3 - GUI + tray**: Fyne settings window and system tray start/stop.
|
|
4. **M4 - Hardening**: config persistence, frame dropping/backpressure, discovery options,
|
|
X11 fallback if feasible, packaging (Nix package / AppImage / release binaries).
|
|
|
|
## Development Environment
|
|
|
|
- Nix flake (`flake.nix`) with `devShell` workspace.
|
|
- `direnv` manages activating the Nix environment.
|
|
- Build: `go build ./...`; Lint: `golangci-lint`; Tests: `go test ./...`.
|
|
- All development follows the rules in `AGENTS.md`.
|
|
|
|
## Open Questions
|
|
|
|
- Bitrate/bandwidth targets beyond the balanced 1080p30 default.
|
|
- Whether to support scaling/downsampling of the captured monitor in M1/M2 or only the
|
|
monitor's native resolution.
|
|
- PipeWire bindings strategy (pure-Go bindings vs. cgo wrappers vs. `xdg-desktop-portal`).
|
|
|
|
## Decisions Made (2026-09-18)
|
|
|
|
1. **Name**: project (and binary) is `teleportfling`.
|
|
2. **JPEG encoding**: match `obs-teleport` exactly - use turbojpeg via cgo (`libturbojpeg`).
|
|
Benchmark against a pure-Go encoder during M1 anyway; use turbojpeg unless it is a
|
|
blocker.
|
|
3. **System tray**: Fyne's native tray support is limited on Linux; evaluate
|
|
`github.com/nicedoc/systray` (or similar) when we reach M3. Do not block M1/M2 on it.
|
|
4. **PipeWire bindings**: unresolved - evaluate pure-Go bindings vs. cgo to libpipewire
|
|
during M1. This drives capture backend build requirements.
|
|
5. **End-to-end testing**: OBS + `obs-teleport` are available locally on the dev machine
|
|
for now (loopback/LAN testing). A remote receiver machine can be added later for
|
|
further testing.
|
|
6. **License**: GPL-2.0, matching the `obs-teleport` project whose protocol we implement.
|
|
|
|
## Decisions Made (2026-09-18, M3/M4)
|
|
|
|
7. **Screen capture backend**: `go2tv.app/screencast` (MIT, GPL-2.0-compatible) via
|
|
`xdg-desktop-portal` + PipeWire. Chosen over hand-rolled cgo bindings. The portal
|
|
path is compositor-agnostic: Wayland (hyprland/gnome portals) and X11
|
|
(`xdg-desktop-portal-gtk`) both work, so a separate native X11/XShm fallback is not
|
|
required for the supported path and was not implemented (project.md M4 "if feasible").
|
|
8. **GUI toolkit**: Fyne v2 — provides both the settings window and a system tray
|
|
(StatusNotifierItem over DBus), so no separate systray dependency was needed.
|
|
9. **Engine decoupling**: streaming core lives in `internal/flinger` (GUI-free) so the
|
|
CLI, GUI and any future daemon share one implementation.
|
|
|
|
## M4 Hardening Status
|
|
|
|
- Config persistence + validation: done (internal/config, flinger.Validate).
|
|
- Backpressure/queue stats: done (sender tracks dropped frames; exposed in engine
|
|
Status and the GUI status label).
|
|
- Discovery on/off: done (flinger.Config.Announce, GUI checkbox).
|
|
- X11 fallback: not implemented — the portal backend already covers X11 sessions.
|
|
- Packaging (Nix package / AppImage): deferred (see AGENTS.md / roadmap). |