Case study Built at Veritone
SRT/HLS Stream Ingestion Engine.
A production recorder for live broadcast streams built to run around the clock, unsupervised, for days at a time.
- Built at
- Veritone
- Stack
- Python · FFmpeg · SRT · HLS · AWS S3 · Kubernetes
- Ran
- 24/7, still in production when I left in 2026
- Soak test
- 53.8 hours continuous on SRT, no drift
- Output
- MP4/TS segments to S3, at a length the user sets
Overview
The problem and my role.
The problem
Veritone needed live broadcast streams recorded around the clock. They arrived over SRT or HLS, and every recording had to be cut into clock-aligned segments and delivered to S3 with nobody watching.
My role
I designed the engine and built it with Quynh Dang, in Python and FFmpeg on Kubernetes.
Scale
At peak it recorded 10 to 50 live streams at once. It was still in production when I left Veritone in April 2026.
Requirements
What it had to get right.
Two ways in
Streams arrived over SRT or HLS. SRT is low-latency UDP that resends lost packets, and HLS is plain HTTP that works through any CDN. Both go through one shared protocol layer, so the rest of the engine doesn’t care which one it got.
Recovery in layers
It retries up to 999 times with exponential backoff, checks the audio for corruption three ways (AAC decoder errors, timestamp jumps, file sizes), and restarts itself, with a rate limit so restarts can’t pile up.
An engineer’s downmix
Detects 5.1 and 7.1 surround and folds it to stereo with an FFmpeg pan filter. If the source is already stereo, it copies the audio untouched.
Architecture
Signal path.
Normalled left to right, like a patchbay.
Input
srt:// or https://*.m3u8, detected from the URL
Protocol
SRTHandler or HLSHandler behind one abstract base class
FFmpeg
Single pass: buffer, flags, encode or copy, segment
Output
MP4/TS segments to S3 at a length the user sets, fresh client per upload
Monitor
AAC errors, timestamp drift, file size; auto-restart
Design notes
Decisions.
Trust the transport, then verify
SRT timestamps arrive pre-synchronized (TSBPD), so
-copytskeeps them.Never let PTS wrap
HLS broadcaster timestamps accumulate forever. Letting FFmpeg regenerate them avoids the MPEG-TS wraparound at about 26.5 hours.
Cut on keyframes only
Segments break on clean GOP boundaries and reset to zero, so no segment opens on a black frame.