Case study Built at Veritone

SRT/HLS Stream Ingestion Engine.

A production recorder for live broadcast streams built to run around the clock, unsupervised, for days at a time.

Built at
Veritone
Stack
Python · FFmpeg · SRT · HLS · AWS S3 · Kubernetes
Ran
24/7, still in production when I left in 2026
Soak test
53.8 hours continuous on SRT, no drift
Output
MP4/TS segments to S3, at a length the user sets

Overview

The problem and my role.

The problem

Veritone needed live broadcast streams recorded around the clock. They arrived over SRT or HLS, and every recording had to be cut into clock-aligned segments and delivered to S3 with nobody watching.

My role

I designed the engine and built it with Quynh Dang, in Python and FFmpeg on Kubernetes.

Scale

At peak it recorded 10 to 50 live streams at once. It was still in production when I left Veritone in April 2026.

Requirements

What it had to get right.

  1. Two ways in

    Streams arrived over SRT or HLS. SRT is low-latency UDP that resends lost packets, and HLS is plain HTTP that works through any CDN. Both go through one shared protocol layer, so the rest of the engine doesn’t care which one it got.

  2. Recovery in layers

    It retries up to 999 times with exponential backoff, checks the audio for corruption three ways (AAC decoder errors, timestamp jumps, file sizes), and restarts itself, with a rate limit so restarts can’t pile up.

  3. An engineer’s downmix

    Detects 5.1 and 7.1 surround and folds it to stereo with an FFmpeg pan filter. If the source is already stereo, it copies the audio untouched.

Architecture

Signal path.

Normalled left to right, like a patchbay.

  1. Input

    srt:// or https://*.m3u8, detected from the URL

  2. Protocol

    SRTHandler or HLSHandler behind one abstract base class

  3. FFmpeg

    Single pass: buffer, flags, encode or copy, segment

  4. Output

    MP4/TS segments to S3 at a length the user sets, fresh client per upload

  5. Monitor

    AAC errors, timestamp drift, file size; auto-restart

Design notes

Decisions.

  1. Trust the transport, then verify

    SRT timestamps arrive pre-synchronized (TSBPD), so -copyts keeps them.

  2. Never let PTS wrap

    HLS broadcaster timestamps accumulate forever. Letting FFmpeg regenerate them avoids the MPEG-TS wraparound at about 26.5 hours.

  3. Cut on keyframes only

    Segments break on clean GOP boundaries and reset to zero, so no segment opens on a black frame.