# TeleportFling - Project Overview

## Summary

TeleportFling is a standalone Linux application that captures a single screen along with
audio from the host machine and transmits the combined stream over a LAN using the
[Teleport protocol](https://github.com/fzwoch/obs-teleport). The teleport stream is received
by another machine running [OBS Studio](https://obsproject.com) with the `obs-teleport`
plugin installed, where it appears as a regular Teleport source.

The project is an OBS/NDI-like replacement for a minimal multi-machine streaming setup:
one machine (the "flinger") produces a screen + audio feed, a second machine aggregates
and streams it. No NDI compatibility of any form.

## Goals

1. Capture a single monitor (screen) + configurable audio source on Linux.
2. Push the stream over the LAN using the Teleport protocol so an OBS + `obs-teleport`
   receiver can consume it with zero plugin-side configuration on the source machine.
3. Ship a standalone, installable application, not an OBS plugin:
   - Full configuration GUI (device, audio source, quality, discovery settings).
   - System tray icon for quick start/stop and status.
   - Works headless-ish: capture/transmit logic is decoupled from the GUI so it can
     later run as a daemon or be scripted.

## Non-Goals

- NDI or DistroAV compatibility.
- Software transcoding beyond the JPEG frame encoding the protocol requires.
- Synchronous multi-screen capture initially (single screen only, per the name).
- Windows/macOS support initially (Linux only).

## Target Environment

| Concern            | Decision                                                       |
|--------------------|----------------------------------------------------------------|
| Language           | Go (same language as `obs-teleport`)                           |
| Display server     | Wayland primary; X11 support if it falls out cheaply           |
| Audio              | Configurable: system/desktop audio (default) or mic/input device |
| Screen capture     | Wayland: PipeWire / xdg-desktop-portal; X11: X11 extension or PipeWire fallback |
| GUI                | Fyne (config window + system tray)                             |
| Discovery          | UDP multicast peer discovery (identical to `obs-teleport`)     |
| Target quality     | Initial release: 1920x1080 @ 30 fps, balanced JPEG quality     |
| Protocol target    | Latest `obs-teleport` release (protocol contract below)        |
| Remote git         | Public self-hosted Gitea at `https://gitea.edley.me/`         |

## Teleport Protocol Contract

Reference implementation: https://github.com/fzwoch/obs-teleport (GPL-2.0).

1. **Peer discovery** - UDP multicast broadcast of a JSON `AnnouncePayload`:
   `{"Name", "Port", "AudioAndVideo", "Version", "Address"}`. The OBS receiver uses
   these announcements to populate its source list. `obs-teleport` uses
   `github.com/schollz/peerdiscovery`.
2. **Transport** - TCP. The sender binds a listener on a configurable port and accepts
   connections from receivers (multiple concurrent receivers supported).
3. **Packet header** (all little-endian):
   - `Header`: `Type [4]byte`, `Timestamp uint64`, `Size int32`.
   - `ImageHeader` (video only): `ColorMatrix [16]float32`, `ColorRangeMin [3]float32`,
     `ColorRangeMax [3]float32` (from OBS video format color parameters).
   - `WaveHeader` (audio only): `Format int32`, `SampleRate int32`, `Speakers int32`,
     `Frames int32`.
4. **Video frames** - `Type = "JPEG"`. Frames are compressed to JPEG (turbojpeg) with
   configurable quality. Payload = header + image header + JPEG bytes.
5. **Audio** - `Type = "WAVE"`. Raw interleaved PCM. Payload = header + wave header +
   PCM samples (16-bit stereo @ 48 kHz typical).
6. **Ordering** - Video frames are queued, encoded in order, and sent in the same order;
   dropped frames are counted as lagged frames. Audio is sent immediately as it arrives.
7. **Compatibility contract** - We must remain byte-compatible with the latest
   `obs-teleport` release so that stock OBS Teleport receivers can discover and decode
   our stream without modification.

## Architecture (proposed)

```
┌────────────────────────────┐     ┌───────────────────────────────┐
│          Fling app          │     │         OBS receiver          │
│  ┌──────────┐  ┌─────────┐  │     │                               │
│  │ Screen   │  │ Audio   │  │     │  ┌─────────────────────────┐  │
│  │ capture  │  │ capture │  │     │  │  obs-teleport source    │  │
│  └────┬─────┘  └────┬────┘  │     │  └───────────┬─────────────┘  │
│       │             │       │     │              │                 │
│       ▼             ▼       │     │              ▼                 │
│  ┌────────────────────────┐  │     │  ┌─────────────────────────┐  │
│  │ Flinger core           │  │     │  │ (decode + render)       │  │
│  │  · JPEG encode (video) │  ├─────┼─▶│  OBS Studio pipeline    │  │
│  │  · interleave (audio)  │  │     │  └─────────────────────────┘  │
│  │  · packetize           │  │     │                               │
│  │  · TCP sender          │  │     │                               │
│  │  · UDP announcer       │  │     │                               │
│  └───────────┬────────────┘  │     │                               │
└──────────────┼──────────────┘     └───────────────────────────────┘
               │ LAN (TCP + UDP multicast)
```

Components:

1. **Core library (`internal/flinger`)** - protocol + capture agnostic:
   - `protocol`: packet types, JPEG encode, WAVE packetize, header (de)serialization.
   - `output`: TCP listener, connection management (reuse `obs-teleport`'s per-connection
     buffered channel approach with drop-when-overflow), framing/ordering queue.
   - `discovery`: UDP multicast announcer (`AnnouncePayload`).
   - `capture`: interface `CaptureSource` + implementations for screen and audio.
2. **GUI (`cmd/teleportfling` or `internal/gui`)** - Fyne:
   - Settings window: monitor picker, audio source picker, port, JPEG quality,
     resolution/fps targets, discovery on/off, enable/disable.
   - System tray: start/stop toggle, status indicator, open settings, quit.
   - Config persistence (JSON config file, e.g. `~/.config/teleportfling/config.json`).
3. **Capture backend**:
   - Wayland: PipeWire via `pipewire` Go bindings or `xdg-desktop-portal` screen capture.
   - Audio: PipeWire/PulseAudio monitor source (system audio) or input device; configurable.
   - X11 (optional): X11 API screen grab or PipeWire fallback where available.

## Milestones

1. **M1 - Protocol proof of life**: static JPEG frames + synthesized audio sent over TCP,
   multicast announcements read successfully by OBS Teleport receiver. CLI only.
2. **M2 - Real capture**: PipeWire screen capture + system audio capture feeding the
   core; confirm 1080p30 on Wayland.
3. **M3 - GUI + tray**: Fyne settings window and system tray start/stop.
4. **M4 - Hardening**: config persistence, frame dropping/backpressure, discovery options,
   X11 fallback if feasible, packaging (Nix package / AppImage / release binaries).

## Development Environment

- Nix flake (`flake.nix`) with `devShell` workspace.
- `direnv` manages activating the Nix environment.
- Build: `go build ./...`; Lint: `golangci-lint`; Tests: `go test ./...`.
- All development follows the rules in `AGENTS.md`.

## Open Questions

- Bitrate/bandwidth targets beyond the balanced 1080p30 default.
- Whether to support scaling/downsampling of the captured monitor in M1/M2 or only the
  monitor's native resolution.
- PipeWire bindings strategy (pure-Go bindings vs. cgo wrappers vs. `xdg-desktop-portal`).

## Decisions Made (2026-09-18)

1. **Name**: project (and binary) is `teleportfling`.
2. **JPEG encoding**: match `obs-teleport` exactly - use turbojpeg via cgo (`libturbojpeg`).
   Benchmark against a pure-Go encoder during M1 anyway; use turbojpeg unless it is a
   blocker.
3. **System tray**: Fyne's native tray support is limited on Linux; evaluate
   `github.com/nicedoc/systray` (or similar) when we reach M3. Do not block M1/M2 on it.
4. **PipeWire bindings**: unresolved - evaluate pure-Go bindings vs. cgo to libpipewire
   during M1. This drives capture backend build requirements.
5. **End-to-end testing**: OBS + `obs-teleport` are available locally on the dev machine
   for now (loopback/LAN testing). A remote receiver machine can be added later for
   further testing.
6. **License**: GPL-2.0, matching the `obs-teleport` project whose protocol we implement.

## Decisions Made (2026-09-18, M3/M4)

7. **Screen capture backend**: `go2tv.app/screencast` (MIT, GPL-2.0-compatible) via
   `xdg-desktop-portal` + PipeWire. Chosen over hand-rolled cgo bindings. The portal
   path is compositor-agnostic: Wayland (hyprland/gnome portals) and X11
   (`xdg-desktop-portal-gtk`) both work, so a separate native X11/XShm fallback is not
   required for the supported path and was not implemented (project.md M4 "if feasible").
8. **GUI toolkit**: Fyne v2 — provides both the settings window and a system tray
   (StatusNotifierItem over DBus), so no separate systray dependency was needed.
9. **Engine decoupling**: streaming core lives in `internal/flinger` (GUI-free) so the
   CLI, GUI and any future daemon share one implementation.

## M4 Hardening Status

- Config persistence + validation: done (internal/config, flinger.Validate).
- Backpressure/queue stats: done (sender tracks dropped frames; exposed in engine
  Status and the GUI status label).
- Discovery on/off: done (flinger.Config.Announce, GUI checkbox).
- X11 fallback: not implemented — the portal backend already covers X11 sessions.
- Packaging (Nix package / AppImage): deferred (see AGENTS.md / roadmap).