TeleportFling/project.md

TeleportFling - Project Overview

Summary

TeleportFling is a standalone Linux application that captures a single screen along with audio from the host machine and transmits the combined stream over a LAN using the Teleport protocol. The teleport stream is received by another machine running OBS Studio with the obs-teleport plugin installed, where it appears as a regular Teleport source.

The project is an OBS/NDI-like replacement for a minimal multi-machine streaming setup: one machine (the "flinger") produces a screen + audio feed, a second machine aggregates and streams it. No NDI compatibility of any form.

Goals

1. Capture a single monitor (screen) + configurable audio source on Linux.
2. Push the stream over the LAN using the Teleport protocol so an OBS + obs-teleport receiver can consume it with zero plugin-side configuration on the source machine.
3. Ship a standalone, installable application, not an OBS plugin:
   • Full configuration GUI (device, audio source, quality, discovery settings).
   • System tray icon for quick start/stop and status.
   • Works headless-ish: capture/transmit logic is decoupled from the GUI so it can later run as a daemon or be scripted.

Non-Goals

• NDI or DistroAV compatibility.
• Software transcoding beyond the JPEG frame encoding the protocol requires.
• Synchronous multi-screen capture initially (single screen only, per the name).
• Windows/macOS support initially (Linux only).

Target Environment

Concern	Decision
Language	Go (same language as obs-teleport)
Display server	Wayland primary; X11 support if it falls out cheaply
Audio	Configurable: system/desktop audio (default) or mic/input device
Screen capture	Wayland: PipeWire / xdg-desktop-portal; X11: X11 extension or PipeWire fallback
GUI	Fyne (config window + system tray)
Discovery	UDP multicast peer discovery (identical to obs-teleport)
Target quality	Initial release: 1920x1080 @ 30 fps, balanced JPEG quality
Protocol target	Latest obs-teleport release (protocol contract below)
Remote git	Self-hosted Gitea at http://homeserver:3050/

Teleport Protocol Contract

Reference implementation: https://github.com/fzwoch/obs-teleport (GPL-2.0).

1. Peer discovery - UDP multicast broadcast of a JSON AnnouncePayload: {"Name", "Port", "AudioAndVideo", "Version", "Address"}. The OBS receiver uses these announcements to populate its source list. obs-teleport uses github.com/schollz/peerdiscovery.
2. Transport - TCP. The sender binds a listener on a configurable port and accepts connections from receivers (multiple concurrent receivers supported).
3. Packet header (all little-endian):
   • Header: Type [4]byte, Timestamp uint64, Size int32.
   • ImageHeader (video only): ColorMatrix [16]float32, ColorRangeMin [3]float32, ColorRangeMax [3]float32 (from OBS video format color parameters).
   • WaveHeader (audio only): Format int32, SampleRate int32, Speakers int32, Frames int32.
4. Video frames - Type = "JPEG". Frames are compressed to JPEG (turbojpeg) with configurable quality. Payload = header + image header + JPEG bytes.
5. Audio - Type = "WAVE". Raw interleaved PCM. Payload = header + wave header + PCM samples (16-bit stereo @ 48 kHz typical).
6. Ordering - Video frames are queued, encoded in order, and sent in the same order; dropped frames are counted as lagged frames. Audio is sent immediately as it arrives.
7. Compatibility contract - We must remain byte-compatible with the latest obs-teleport release so that stock OBS Teleport receivers can discover and decode our stream without modification.

Architecture (proposed)

┌────────────────────────────┐     ┌───────────────────────────────┐
│          Fling app          │     │         OBS receiver          │
│  ┌──────────┐  ┌─────────┐  │     │                               │
│  │ Screen   │  │ Audio   │  │     │  ┌─────────────────────────┐  │
│  │ capture  │  │ capture │  │     │  │  obs-teleport source    │  │
│  └────┬─────┘  └────┬────┘  │     │  └───────────┬─────────────┘  │
│       │             │       │     │              │                 │
│       ▼             ▼       │     │              ▼                 │
│  ┌────────────────────────┐  │     │  ┌─────────────────────────┐  │
│  │ Flinger core           │  │     │  │ (decode + render)       │  │
│  │  · JPEG encode (video) │  ├─────┼─▶│  OBS Studio pipeline    │  │
│  │  · interleave (audio)  │  │     │  └─────────────────────────┘  │
│  │  · packetize           │  │     │                               │
│  │  · TCP sender          │  │     │                               │
│  │  · UDP announcer       │  │     │                               │
│  └───────────┬────────────┘  │     │                               │
└──────────────┼──────────────┘     └───────────────────────────────┘
               │ LAN (TCP + UDP multicast)

Components:

1. Core library (internal/flinger) - protocol + capture agnostic:
   • protocol: packet types, JPEG encode, WAVE packetize, header (de)serialization.
   • output: TCP listener, connection management (reuse obs-teleport's per-connection buffered channel approach with drop-when-overflow), framing/ordering queue.
   • discovery: UDP multicast announcer (AnnouncePayload).
   • capture: interface CaptureSource + implementations for screen and audio.
2. GUI (cmd/teleportfling or internal/gui) - Fyne:
   • Settings window: monitor picker, audio source picker, port, JPEG quality, resolution/fps targets, discovery on/off, enable/disable.
   • System tray: start/stop toggle, status indicator, open settings, quit.
   • Config persistence (JSON config file, e.g. ~/.config/teleportfling/config.json).
3. Capture backend:
   • Wayland: PipeWire via pipewire Go bindings or xdg-desktop-portal screen capture.
   • Audio: PipeWire/PulseAudio monitor source (system audio) or input device; configurable.
   • X11 (optional): X11 API screen grab or PipeWire fallback where available.

Milestones

1. M1 - Protocol proof of life: static JPEG frames + synthesized audio sent over TCP, multicast announcements read successfully by OBS Teleport receiver. CLI only.
2. M2 - Real capture: PipeWire screen capture + system audio capture feeding the core; confirm 1080p30 on Wayland.
3. M3 - GUI + tray: Fyne settings window and system tray start/stop.
4. M4 - Hardening: config persistence, frame dropping/backpressure, discovery options, X11 fallback if feasible, packaging (Nix package / AppImage / release binaries).

Development Environment

• Nix flake (flake.nix) with devShell workspace.
• direnv manages activating the Nix environment.
• Build: go build ./...; Lint: golangci-lint; Tests: go test ./....
• All development follows the rules in AGENTS.md.

Open Questions

• Bitrate/bandwidth targets beyond the balanced 1080p30 default.
• Whether to support scaling/downsampling of the captured monitor in M1/M2 or only the monitor's native resolution.
• PipeWire bindings strategy (pure-Go bindings vs. cgo wrappers vs. xdg-desktop-portal).

Decisions Made (2026-09-18)

1. Name: project (and binary) is teleportfling.
2. JPEG encoding: match obs-teleport exactly - use turbojpeg via cgo (libturbojpeg). Benchmark against a pure-Go encoder during M1 anyway; use turbojpeg unless it is a blocker.
3. System tray: Fyne's native tray support is limited on Linux; evaluate github.com/nicedoc/systray (or similar) when we reach M3. Do not block M1/M2 on it.
4. PipeWire bindings: unresolved - evaluate pure-Go bindings vs. cgo to libpipewire during M1. This drives capture backend build requirements.
5. End-to-end testing: OBS + obs-teleport are available locally on the dev machine for now (loopback/LAN testing). A remote receiver machine can be added later for further testing.
6. License: GPL-2.0, matching the obs-teleport project whose protocol we implement.

Decisions Made (2026-09-18, M3/M4)

7. Screen capture backend: go2tv.app/screencast (MIT, GPL-2.0-compatible) via xdg-desktop-portal + PipeWire. Chosen over hand-rolled cgo bindings. The portal path is compositor-agnostic: Wayland (hyprland/gnome portals) and X11 (xdg-desktop-portal-gtk) both work, so a separate native X11/XShm fallback is not required for the supported path and was not implemented (project.md M4 "if feasible").
8. GUI toolkit: Fyne v2 — provides both the settings window and a system tray (StatusNotifierItem over DBus), so no separate systray dependency was needed.
9. Engine decoupling: streaming core lives in internal/flinger (GUI-free) so the CLI, GUI and any future daemon share one implementation.

M4 Hardening Status

• Config persistence + validation: done (internal/config, flinger.Validate).
• Backpressure/queue stats: done (sender tracks dropped frames; exposed in engine Status and the GUI status label).
• Discovery on/off: done (flinger.Config.Announce, GUI checkbox).
• X11 fallback: not implemented — the portal backend already covers X11 sessions.
• Packaging (Nix package / AppImage): deferred (see AGENTS.md / roadmap).