feat: add PipeWire screen and system audio capture (M2)
Capture the Wayland desktop via xdg-desktop-portal + PipeWire using go2tv.app/screencast (MIT), and stream it to OBS: - internal/capture: Capture/FrameSource/AudioSource interfaces and the PipeWire backend (BGRA frames at monitor resolution, S16 48 kHz stereo system audio) - protocol: EncodeBGRA fast path producing 4:2:0 YCbCr JPEGs - cmd: --source screen|pattern, --audio, --stream-index flags; real capture feeds the existing sender - share one wall-clock reference between the audio and video loops so OBS receives aligned A/V timestamps (avoids multi-second latency) Verified end-to-end: real desktop at 30 fps renders in OBS with sub-second latency.
This commit is contained in:
@@ -0,0 +1,85 @@
|
||||
// Package capture abstracts the screen and audio sources that feed the
|
||||
// Teleport sender. Implementations are platform-specific; the PipeWire
|
||||
// backend (pipewire.go) handles Wayland via xdg-desktop-portal.
|
||||
//
|
||||
// The core only knows about two things: a stream of video frames and a
|
||||
// stream of audio samples. Everything else (formats, portal negotiation)
|
||||
// stays behind this interface so the sender can later run headless or be
|
||||
// driven by a GUI without coupling.
|
||||
package capture
|
||||
|
||||
import (
|
||||
"io"
|
||||
"time"
|
||||
)
|
||||
|
||||
// VideoFrame is one captured screen frame. Pix holds BGRA (blue, green,
|
||||
// red, alpha) bytes in row-major order; Stride is the byte offset between
|
||||
// consecutive rows.
|
||||
type VideoFrame struct {
|
||||
Pix []byte
|
||||
Width int
|
||||
Height int
|
||||
Stride int
|
||||
}
|
||||
|
||||
// Capture is the combined screen + audio source. Close releases the
|
||||
// underlying capture session (and portal resources).
|
||||
type Capture interface {
|
||||
// Video returns the frame source. Frames arrive at the compositor's
|
||||
// refresh rate and are consumed one at a time via NextFrame.
|
||||
Video() FrameSource
|
||||
// Audio returns the audio source, or nil if audio capture is disabled
|
||||
// or unavailable.
|
||||
Audio() AudioSource
|
||||
io.Closer
|
||||
}
|
||||
|
||||
// FrameSource yields consecutive captured video frames.
|
||||
type FrameSource interface {
|
||||
// NextFrame blocks until the next frame is available and returns it.
|
||||
NextFrame() (*VideoFrame, error)
|
||||
}
|
||||
|
||||
// AudioSource yields raw interleaved PCM samples (signed 16-bit
|
||||
// little-endian, 48 kHz, stereo) read from the system's default output.
|
||||
type AudioSource interface {
|
||||
io.ReadCloser
|
||||
}
|
||||
|
||||
// ErrNoAudio is returned when the underlying backend cannot provide system
|
||||
// audio capture (e.g. sandboxed Flatpak without a direct PipeWire link).
|
||||
var ErrNoAudio = &AudioUnavailableError{}
|
||||
|
||||
// AudioUnavailableError signals that audio capture could not be started.
|
||||
type AudioUnavailableError struct{}
|
||||
|
||||
func (e *AudioUnavailableError) Error() string {
|
||||
return "capture: system audio unavailable"
|
||||
}
|
||||
|
||||
// silenceStep is the pacing interval between silence chunk reads.
|
||||
const silenceStep = 10 * time.Millisecond
|
||||
|
||||
// SilenceSource yields a continuous stream of digital silence, paced like a
|
||||
// real audio capture. It keeps OBS's audio pipeline alive when no system
|
||||
// audio is available or a synthetic source is in use.
|
||||
type silenceSource struct{}
|
||||
|
||||
// NewSilenceSource creates a silence-generating audio source.
|
||||
func NewSilenceSource() AudioSource {
|
||||
return &silenceSource{}
|
||||
}
|
||||
|
||||
func (s *silenceSource) Read(p []byte) (int, error) {
|
||||
if len(p) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
// Emit a chunk of silence at a real-audio cadence. The reader is 1:1
|
||||
// stereo S16, so 48000 * 10ms * 2ch * 2 bytes = 1920 bytes per read.
|
||||
clear(p)
|
||||
time.Sleep(silenceStep)
|
||||
return len(p), nil
|
||||
}
|
||||
|
||||
func (s *silenceSource) Close() error { return nil }
|
||||
Reference in New Issue
Block a user