By Maya Chen · 2026-10-23
Important: Download only videos you have the right or authorization to download. You are responsible for complying with applicable copyright law, platform terms, privacy requirements, and other applicable rules.
yt-dlp is one of the most capable open-source tools for downloading and processing online video. What begins as a simple command-line utility can become a surprisingly sophisticated part of a video ingestion pipeline.
Running:
yt-dlp "VIDEO_URL"
is easy.
Running yt-dlp reliably across thousands or millions of URLs is a completely different engineering problem.
At scale, you need to think about extraction, JavaScript runtimes, FFmpeg, proxy infrastructure, retries, queues, storage, deduplication, observability, container isolation, concurrency limits, authentication, updates, and failures caused by upstream platform changes.
This guide covers both sides of the problem:
- how to install and use yt-dlp correctly in 2026;
- how to design a production architecture around it;
- how proxies fit into that architecture;
- how to containerize workers;
- how to organize downloaded media;
- how to test and monitor the system;
- and when it makes more sense to use a managed video-download service such as VideoScale instead of maintaining the entire stack yourself.
«Important: Download only videos you have the right or authorization to download. You are responsible for complying with applicable copyright law, platform terms, privacy requirements, and other applicable rules.»
Understand yt-dlp in 2026
What Is yt-dlp?
yt-dlp is a feature-rich command-line audio/video downloader supporting thousands of websites.
It originated as a fork of youtube-dl and has evolved into an actively maintained project with a large extractor ecosystem, extensive format-selection capabilities, metadata extraction, subtitles, thumbnails, authentication, post-processing, proxy support, browser impersonation, plugins, and integrations with external downloaders.
For YouTube specifically, yt-dlp can handle tasks including:
- individual videos;
- playlists;
- channels;
- Shorts;
- live streams;
- audio-only extraction;
- subtitles;
- automatically generated subtitles;
- thumbnails;
- metadata;
- format selection;
- separate audio/video streams;
- FFmpeg merging;
- cookies;
- proxies;
- output templates;
- download archives;
- filtering;
- post-processing;
- structured JSON metadata output.
That makes it useful both as an interactive CLI and as a building block for larger applications.
What Changed in yt-dlp in 2026?
One of the most important operational lessons with yt-dlp is that keeping it current is part of running it.
YouTube and other platforms change continuously. Extraction logic that works today can stop working after an upstream change.
As of August 22, 2026, the latest upstream release is 2026.08.19.
Recent development has included several changes worth understanding.
August 2026: YouTube Client and Extraction Improvements
The August 19 release added and adjusted several YouTube-related capabilities, including:
- a "visionos" YouTube player client;
- "web_embedded" client fallbacks;
- updated player client versions;
- player-client maintenance;
- improvements to channel metadata extraction;
- fixes for live adaptive fragment generation;
- removal of "android_vr" from the default client set.
The same release also added support for "curl_cffi" 0.16.x and continued work around browser impersonation.
These changes illustrate why a production installation should not treat yt-dlp as a binary you install once and forget.
Python 3.11 Is Now the Recommended Baseline
The July 2026 release raised the minimum recommended Python version to Python 3.11.
Python 3.10 was approaching end-of-life, and upstream indicated that support would be removed in the future.
For a new server deployment, therefore, Python 3.11+ is the sensible baseline.
JavaScript Runtime Requirements Matter More
Modern YouTube extraction increasingly involves JavaScript execution.
Current yt-dlp documentation strongly recommends:
- "ffmpeg" / "ffprobe";
- "yt-dlp-ejs" for full YouTube support;
- a supported JavaScript runtime.
Deno is the recommended runtime.
Recent requirements raised the minimum Deno version to 2.3.0, while supported Node.js versions are 22+. Bun support has been deprecated and restricted to specific versions.
For a fresh deployment, use Deno unless you have a reason to standardize on Node.js.
Security Hardening
Several 2026 releases contained security changes affecting features including:
- "--netrc-cmd";
- "--exec";
- external curl downloading;
- aria2c;
- filename handling;
- shortcut/link output.
Notably, HLS/DASH downloading through aria2c was removed following a security issue. Native concurrent fragment downloading using "-N" is now the better direction for those workflows.
This matters for production systems because blindly copying old yt-dlp command examples from blog posts can introduce unnecessary risk.
Build the Ubuntu download stack
Recommended Ubuntu Architecture
For a basic server, the stack looks like:
Application
|
v
yt-dlp
|
+---- Deno / JavaScript runtime
|
+---- FFmpeg / ffprobe
|
+---- Proxy
|
v
YouTube
|
v
Temporary Storage
|
v
Final Storage / Object Storage
At scale, a better architecture looks like:
+--------------------+
Client ---------->| API / Control Plane|
+---------+----------+
|
v
+--------------------+
| Job Queue |
| Redis / RabbitMQ |
+---------+----------+
|
+--------------+--------------+
| | |
v v v
Worker 1 Worker 2 Worker N
| | |
+------- Proxy Layer ---------+
|
v
YouTube
|
v
Temporary Storage
|
v
S3 / R2 / MinIO / NAS
Each worker should ideally be disposable.
That is why containers are attractive for production deployments.
Installing yt-dlp on Ubuntu
The exact Ubuntu package versions depend on the Ubuntu release, but the following is a good general-purpose setup.
Start by updating the machine:
sudo apt update
sudo apt upgrade -y
Install basic utilities:
sudo apt install -y \
curl
wget
git
ca-certificates
gnupg
unzip
jq
ffmpeg
python3
python3-pip
python3-venv
Check Python:
python3 --version
For a new 2026 installation, target Python 3.11 or later.
Check FFmpeg:
ffmpeg -version
And:
ffprobe -version
Installing yt-dlp with pip
Using an isolated Python environment avoids polluting the system Python installation.
Create a directory:
sudo mkdir -p /opt/yt-dlp
sudo chown "$USER":"$USER" /opt/yt-dlp
cd /opt/yt-dlp
Create a virtual environment:
python3 -m venv venv
Activate it:
source venv/bin/activate
Upgrade packaging tools:
python -m pip install --upgrade pip setuptools wheel
Install yt-dlp with the recommended dependency set and browser-impersonation support:
python -m pip install -U "yt-dlp[default,curl-cffi]"
Check installation:
yt-dlp --version
Inspect verbose environment information:
yt-dlp --verbose
List extractors:
yt-dlp --list-extractors
List available browser impersonation targets:
yt-dlp --list-impersonate-targets
Installing Deno
For full modern YouTube support, provide a compatible JavaScript runtime.
Deno is currently the recommended choice.
After installing Deno using its official installation instructions, verify it with:
deno --version
Use Deno 2.3.0 or later with current yt-dlp versions.
If you prefer Node.js:
node --version
Current yt-dlp documentation requires supported Node versions to be Node 22 or newer.
Alternative: Official yt-dlp Binary
Linux users can also use the project's release binary.
A typical layout is:
sudo install -d /usr/local/bin
Place the official "yt-dlp" release there and make it executable:
sudo chmod a+rx /usr/local/bin/yt-dlp
Then:
yt-dlp --version
Official release binaries support:
yt-dlp -U
yt-dlp currently maintains three release channels:
stable nightly master
Interestingly, upstream currently recommends nightly for regular users, because websites change frequently and stable releases can become outdated between releases.
Switch to nightly with:
yt-dlp --update-to nightly
For production, however, consider testing a new build before deploying it across every worker.
A practical model is:
nightly | v canary worker | v integration tests | v production worker image
Basic yt-dlp Commands
Download a video:
yt-dlp "VIDEO_URL"
List available formats:
yt-dlp -F "VIDEO_URL"
Download the automatically selected best combination:
yt-dlp "VIDEO_URL"
Explicitly request best video plus best audio:
yt-dlp -f "bestvideo+bestaudio/best" "VIDEO_URL"
Prefer MP4-compatible streams:
yt-dlp \
-f "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4]/best"
"VIDEO_URL"
Merge into MP4 when possible:
yt-dlp \
-f "bestvideo+bestaudio/best"
--merge-output-format mp4
"VIDEO_URL"
Limit Video Resolution
Up to 1080p:
yt-dlp \
-f "bv*[height<=1080]+ba/b[height<=1080]"
"VIDEO_URL"
Up to 720p:
yt-dlp \
-f "bv*[height<=720]+ba/b[height<=720]"
"VIDEO_URL"
This is useful when storage and bandwidth matter more than retaining the absolute highest available quality.
Audio Downloads
Extract audio:
yt-dlp -x "VIDEO_URL"
Convert to MP3:
yt-dlp \
-x
--audio-format mp3
"VIDEO_URL"
High-quality MP3:
yt-dlp \
-x
--audio-format mp3
--audio-quality 0
"VIDEO_URL"
Other supported audio conversion formats include:
aac alac flac m4a mp3 opus vorbis wav
FFmpeg and ffprobe are required for these post-processing workflows.
Metadata
Print metadata as JSON without downloading:
yt-dlp -j "VIDEO_URL"
Pretty-print it:
yt-dlp -j "VIDEO_URL" | jq .
For playlist-level structured information:
yt-dlp -J "PLAYLIST_URL"
Write metadata alongside the video:
yt-dlp \
--write-info-json
"VIDEO_URL"
For data pipelines, metadata extraction is often as important as the video itself.
You can store:
video.mp4 video.info.json video.jpg video.en.vtt
together as one logical media object.
Thumbnails
Download the thumbnail:
yt-dlp \
--write-thumbnail
"VIDEO_URL"
Convert thumbnails:
yt-dlp \
--write-thumbnail
--convert-thumbnails jpg
"VIDEO_URL"
Embed the thumbnail where supported:
yt-dlp \
--embed-thumbnail
"VIDEO_URL"
Subtitles
List subtitles:
yt-dlp --list-subs "VIDEO_URL"
Download subtitles:
yt-dlp \
--write-subs
"VIDEO_URL"
Download automatically generated subtitles:
yt-dlp \
--write-auto-subs
"VIDEO_URL"
Select English:
yt-dlp \
--write-subs
--sub-langs "en.*"
"VIDEO_URL"
Convert them to SRT:
yt-dlp \
--write-subs
--sub-langs "en.*"
--convert-subs srt
"VIDEO_URL"
For AI pipelines, subtitles can be particularly useful because they may eliminate the need to transcribe the entire audio stream yourself.
Output Templates
yt-dlp's output templating system is extremely powerful.
Simple example:
yt-dlp \
-o "%(title)s.%(ext)s"
"VIDEO_URL"
Include the video ID:
yt-dlp \
-o "%(title)s [%(id)s].%(ext)s"
"VIDEO_URL"
For a channel archive:
yt-dlp \
-o "%(channel)s/%(upload_date)s - %(title)s [%(id)s].%(ext)s"
"URL"
For large-scale ingestion, IDs are preferable to titles as canonical identifiers.
A good storage layout is:
/videos/ youtube/ CHANNEL_ID/ VIDEO_ID/ source.mp4 metadata.json thumbnail.jpg subtitles/
Titles can change.
YouTube video IDs are far better database/storage keys.
Download Archives and Deduplication
One of the most important yt-dlp options for recurring jobs is:
--download-archive
Example:
yt-dlp \
--download-archive /data/archive.txt
"CHANNEL_URL"
After successfully downloading an item, yt-dlp records it.
On future executions it can skip previously processed videos.
For production architectures, use both:
yt-dlp archive
database-level uniqueness
For example, your database can enforce:
UNIQUE(platform, video_id)
The database should be the authoritative source of truth; the archive can remain a useful worker-level optimization.
Date Filtering
Download videos after a particular date:
yt-dlp \
--dateafter 20260101
"CHANNEL_URL"
Before a date:
yt-dlp \
--datebefore 20260701
"CHANNEL_URL"
Relative date syntax is also supported.
This is useful for incremental channel ingestion.
Filtering
yt-dlp provides "--match-filters".
For example:
yt-dlp \
--match-filters "!is_live"
"URL"
Filters can reference output-template fields and support combinations of conditions.
At scale, however, consider separating:
metadata discovery | v business filtering | v download
rather than encoding every product rule into a giant shell command.
Using Cookies
Some content may require authentication or an established browser session.
yt-dlp supports:
--cookies FILE
For example:
yt-dlp \
--cookies /secure/cookies.txt
"VIDEO_URL"
It can also read cookies from supported browsers:
yt-dlp \
--cookies-from-browser chrome
"VIDEO_URL"
Supported browsers include Chrome, Chromium, Firefox, Edge, Brave, Opera, Safari, Vivaldi, and others.
Be extremely careful with cookies.
Treat a cookie file like a credential.
Never:
commit it to Git put it inside a public Docker image print it into application logs store it in an unsecured bucket send it to an AI coding assistant
Use a secrets manager or securely mounted secret instead.
Network, test, and protect downloads
Using Proxies with yt-dlp
yt-dlp has native proxy support:
yt-dlp \
--proxy "http://USER:PASSWORD@HOST:PORT"
"VIDEO_URL"
HTTPS-style proxy endpoint:
yt-dlp \
--proxy "https://USER:PASSWORD@HOST:PORT"
"VIDEO_URL"
SOCKS5:
yt-dlp \
--proxy "socks5://USER:PASSWORD@HOST:PORT"
"VIDEO_URL"
A proxy can also be used specifically for geo verification using:
--geo-verification-proxy
Do not hardcode proxy passwords into source code.
Use environment variables or a secret manager.
For example:
export YTDLP_PROXY='http://USER:PASSWORD@HOST:PORT'
Then:
yt-dlp \
--proxy "$YTDLP_PROXY"
"VIDEO_URL"
What Kind of Proxy Should You Use?
For small workloads, you may not need a proxy at all.
For distributed video-processing infrastructure, however, egress management becomes an architectural concern.
Typical proxy categories are:
Datacenter Static residential Rotating residential ISP/static ISP
Datacenter proxies are generally cheaper and faster, but their IP ranges are easier for platforms to classify as infrastructure traffic.
Residential proxies use residential-network IP addresses.
For workloads where you legitimately need distributed residential egress or geographic coverage, a provider such as "residential proxy" (https://privateproxy.me/residential-proxy) offers rotating and static residential endpoints.
PrivateProxy advertises rotating residential endpoints with IP rotation per request or timer, sticky sessions, HTTP(S) and SOCKS5 connectivity, and country/city targeting.
For long-running video downloads, session consistency matters. Rotating an IP in the middle of one transfer can break a session or produce inconsistent behavior.
Therefore, architect proxy selection around a job/session, not blindly around every HTTP request.
A useful abstraction is:
Download Job | v Proxy Assignment | +--- session / endpoint | v
yt-dlp worker
For longer downloads, sticky sessions or an appropriately stable endpoint are usually preferable to uncontrolled per-request rotation.
Proxy Rotation and Warming
At significant scale, do not let every worker invent its own proxy strategy.
Create a proxy-management layer:
Proxy Manager
|
+---------------+---------------+
| | |
Pool A Pool B Pool C
| | |
Worker 1 Worker 2 Worker 3
The proxy manager can track operational signals such as:
provider region session assigned_at success_count failure_count latency cooldown_until health
When using PrivateProxy rotating residential infrastructure, the provider performs IP rotation at the proxy-network layer according to the endpoint/session configuration. Your application can add another layer above it: assign sessions to jobs, test endpoint health, maintain cooldown policies, and avoid repeatedly sending failed jobs through the same route.
“Proxy warming” should not mean trying to disguise abusive automation. In a legitimate infrastructure context, it should mean conservative health validation before assigning production traffic: establish connectivity, verify expected region/egress behavior, and ramp workload gradually rather than immediately saturating a newly provisioned route.
Testing a Proxy
Before involving yt-dlp, validate connectivity independently.
For example:
curl \
--proxy "$YTDLP_PROXY" \
https://example.com/
Then test yt-dlp without downloading:
yt-dlp \
--proxy "$YTDLP_PROXY"
--simulate
"VIDEO_URL"
For verbose diagnostics:
yt-dlp \
--verbose
--proxy "$YTDLP_PROXY"
--simulate
"VIDEO_URL"
Never paste verbose production logs publicly without checking them for credentials, cookies, URLs containing tokens, or other sensitive information.
Browser Impersonation
Modern anti-bot systems can consider TLS and HTTP fingerprints in addition to the User-Agent header.
yt-dlp supports request impersonation when the appropriate dependency is installed.
List targets:
yt-dlp --list-impersonate-targets
Example:
yt-dlp \
--impersonate chrome
"VIDEO_URL"
Upstream currently recommends "curl_cffi" for browser-request impersonation.
Install it through the yt-dlp extra:
pip install -U "yt-dlp[default,curl-cffi]"
Do not automatically force impersonation on every request. The yt-dlp documentation explicitly notes that doing so can hurt download speed or stability.
Concurrent Fragment Downloads
yt-dlp provides:
-N
for concurrent fragment downloads.
For example:
yt-dlp \
-N 4
"VIDEO_URL"
Higher is not automatically better.
If you have:
20 workers × 8 simultaneous videos × 8 fragments
you may unexpectedly create:
1,280 concurrent fragment requests
before accounting for metadata requests, retries, thumbnails, subtitles, and other traffic.
Concurrency therefore needs to be managed globally.
Rate Limiting
yt-dlp provides controls that can help prevent individual workers from consuming unlimited bandwidth.
At the infrastructure level, consider limiting:
jobs per worker fragments per job bandwidth per worker jobs per proxy session retry frequency global concurrency
The correct architecture is usually conservative and adaptive rather than simply maximizing parallelism.
Retries
Networks fail.
Proxy endpoints fail.
CDNs fail.
Videos disappear.
Your queue should assume that failures are normal.
A basic retry strategy might look like:
Attempt 1 | failure | wait | Attempt 2 | failure | longer wait | Attempt 3 | failure | dead-letter queue
Use exponential backoff with jitter at the orchestration layer.
Do not create infinite retry loops.
Testing yt-dlp Without Downloading
One of the most useful production health checks is:
yt-dlp \
--simulate
"VIDEO_URL"
You can inspect formats:
yt-dlp \
--simulate
--list-formats
"VIDEO_URL"
Metadata:
yt-dlp \
--dump-json
"VIDEO_URL"
Verbose diagnostics:
yt-dlp \
--verbose
--simulate
"VIDEO_URL"
A worker readiness test should verify:
yt-dlp executable
Python FFmpeg
ffprobe
JavaScript runtime DNS outbound HTTPS proxy connectivity if configured temporary storage final storage queue connection
Production Health-Check Script
Example:
#!/usr/bin/env bash
set -euo pipefail
echo "Checking yt-dlp..."
yt-dlp --version
echo "Checking FFmpeg..."
ffmpeg -version >/dev/null
echo "Checking ffprobe..."
ffprobe -version >/dev/null
echo "Checking Deno..."
deno --version
echo "Checking disk..."
df -h
echo "Checking yt-dlp extraction..."
yt-dlp \
--simulate
--no-warnings
"$HEALTHCHECK_VIDEO_URL"
echo "Worker healthy."
Use a video you are authorized to access as your health-check target.
Operate workers at scale
Dockerizing yt-dlp
For production-scale deployments, containers provide several advantages:
- reproducible dependencies;
- immutable workers;
- easier rollbacks;
- resource limits;
- horizontal scaling;
- consistent FFmpeg versions;
- isolated updates.
A conceptual Dockerfile:
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
RUN apt-get update \
&& apt-get install -y \
ffmpeg \
curl \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir \
"yt-dlp[default,curl-cffi]"
WORKDIR /app
COPY worker.py /app/worker.py
RUN mkdir -p /downloads /tmp/yt-dlp
CMD ["python", "/app/worker.py"]
For full YouTube support, add a supported JS runtime such as Deno to the image and validate its version during the image build.
Do not use:
latest
everywhere in a serious production deployment.
Pin known-good versions or immutable image digests where practical.
Docker Compose Architecture
A small deployment might contain:
services:
redis:
image: redis:7-alpine
restart: unless-stopped
worker:
build: ./worker
restart: unless-stopped
depends_on:
- redis
environment:
REDIS_URL: redis://redis:6379/0
STORAGE_PATH: /downloads
volumes:
- ./downloads:/downloads
For multiple workers:
docker compose up -d --scale worker=5
For serious scale, Kubernetes, Nomad, ECS or another orchestrator is usually more appropriate than a single Docker Compose host.
Recommended Worker Model
Avoid running yt-dlp directly inside your public HTTP request handler.
Bad architecture:
POST /download
|
v
yt-dlp runs for 20 minutes
|
HTTP request waits
Better:
POST /downloads
|
v
validate request | v create job | v return job_id | v queue | v worker
The API might immediately return:
{
"job_id": "job_abc123", "status": "queued"
}
The client then polls:
GET /downloads/job_abc123
or receives a webhook when processing completes.
Queue Architecture
A production queue can use:
Redis + BullMQ Redis + RQ Celery RabbitMQ AWS SQS Google Pub/Sub Kafka
You probably do not need Kafka just to run yt-dlp.
A Redis-based queue is sufficient for many installations.
Jobs should contain references, not secrets:
{
"job_id": "job_123", "source": "youtube", "url": "...", "format_policy": "1080p", "storage_key": "..."
}
Proxy credentials should be retrieved from a secrets system by the worker.
Storage Architecture
Downloading large video files directly onto the root filesystem is a common mistake.
A worker should have temporary scratch space:
/tmp/video-jobs/JOB_ID/
For example:
/tmp/video-jobs/abc123/video.webm /tmp/video-jobs/abc123/audio.m4a /tmp/video-jobs/abc123/final.mp4
After processing:
upload final object verify upload write metadata mark job complete delete temporary files
Object Storage
For distributed systems, object storage is usually preferable to a shared local disk.
Common choices include:
Amazon S3 Cloudflare R2 Backblaze B2 Google Cloud Storage MinIO
Example logical keys:
youtube/ CHANNEL_ID/ VIDEO_ID/ original.mp4 metadata.json thumbnail.jpg subtitles/ en.vtt
Another model is:
videos/{video_id}/source.mp4 videos/{video_id}/metadata.json videos/{video_id}/thumbnail.jpg
Keep the database separate from the object payload.
Database:
video_id platform source_url channel_id title duration status storage_key size_bytes checksum created_at completed_at
Object storage:
actual multi-megabyte/gigabyte media
Temporary Disk Capacity
Disk exhaustion is one of the easiest ways to kill a downloader fleet.
Suppose a worker processes four simultaneous 4 GB videos.
During processing you may temporarily hold:
video stream audio stream merged file
That means actual temporary usage can significantly exceed the final output size.
Workers should monitor:
df -h
and refuse new jobs below a configured free-space threshold.
For example:
MIN_FREE_SPACE=20GB
Checksums
After upload, calculate a checksum when integrity matters:
sha256sum final.mp4
Store it in your database.
This gives you:
integrity verification deduplication assistance corruption detection
Observability
A production downloader needs metrics.
Useful metrics include:
jobs_queued jobs_started jobs_completed jobs_failed download_duration_seconds download_bytes download_speed proxy_failures extractor_failures
ffmpeg_failures
storage_failures retry_count worker_disk_free_bytes
Track failure categories separately.
For example:
NETWORK_ERROR EXTRACTION_ERROR AUTH_ERROR VIDEO_UNAVAILABLE RATE_LIMITED PROXY_ERROR FFMPEG_ERROR STORAGE_ERROR INTERNAL_ERROR
Otherwise your monitoring dashboard will simply say:
12% failed
without telling you what actually broke.
Logging
Give every job a correlation ID:
job_id=abc123
Every worker log should contain it:
2026-08-22T12:00:01Z job_id=abc123 status=started 2026-08-22T12:00:02Z job_id=abc123 stage=metadata 2026-08-22T12:00:04Z job_id=abc123 stage=download 2026-08-22T12:02:31Z job_id=abc123 stage=upload 2026-08-22T12:02:38Z job_id=abc123 status=complete
Redact:
proxy passwords cookies Authorization headers signed storage URLs session tokens
Updating yt-dlp Safely
Never automatically update every production worker simultaneously.
Use:
new yt-dlp version | v build container | v unit tests | v integration tests | v canary worker | v production rollout
Test several representative cases:
normal video high-resolution video video with separate audio/video playlist Short live-related metadata subtitles thumbnail proxy path cookie-authenticated path if applicable
If extraction breaks after deployment:
rollback image
This is one of the major advantages of immutable Docker images.
Design and secure production infrastructure
Production Architecture
A more complete deployment can look like:
Internet
|
v
+-------------+
| API Gateway |
+------+------+
|
v
+-------------+
| API Service |
+------+------+
|
+--------------+--------------+
| |
v v
PostgreSQL Redis Queue
|
+--------------------+--------------------+
| | |
v v v
Worker Worker Worker
| | |
+----------+---------+---------+----------+
|
v
Proxy Manager
|
v
Proxy Provider
|
v
YouTube
Workers:
|
+--> FFmpeg
|
+--> temporary SSD
|
+--> Object Storage
Monitoring:
Prometheus / Grafana / logs / alerts
Security Model
Treat submitted URLs as untrusted input.
Do not construct shell commands like:
os.system("yt-dlp " + user_url)
That creates command-injection risk.
Instead use argument arrays:
subprocess.run([ "yt-dlp", "--no-playlist", user_url ])
Better yet, use yt-dlp's Python API when appropriate.
Also:
- restrict supported URL schemes;
- consider allowlisting supported domains if the service is YouTube-only;
- never enable "file://" URLs for arbitrary users;
- isolate workers;
- run workers as non-root;
- restrict filesystem access;
- restrict cloud IAM permissions;
- keep secrets outside images;
- cap CPU, memory, disk and execution time.
AI Prompt: Build the Complete Architecture
The following prompt can be given to Cursor, Claude Code, Replit Agent, or another coding agent.
You are a senior infrastructure engineer.
Build a production-ready distributed video ingestion service around yt-dlp.
GOALS
Create an API that accepts authorized YouTube video URLs, creates asynchronous jobs, downloads and processes media using isolated workers, and stores resulting media in S3-compatible object storage.
TECH STACK
- Ubuntu Linux
- Docker
- Docker Compose for local/single-server deployment
- Python 3.12
- FastAPI
- PostgreSQL
- Redis
- yt-dlp
- FFmpeg
- Deno
- S3-compatible object storage
- Prometheus metrics
- structured JSON logging
ARCHITECTURE
API:
- POST /v1/downloads
- GET /v1/downloads/{job_id}
- DELETE /v1/downloads/{job_id}
- GET /health
- GET /ready
- GET /metrics
Workers:
- consume jobs from Redis
- use yt-dlp
- use FFmpeg
- download to isolated temporary directories
- upload completed objects to S3
- delete temporary files after successful upload
- retry transient failures with exponential backoff
- send permanent failures to a dead-letter queue
SECURITY
- never execute user input through shell=True
- validate URLs
- only allow http and https
- optionally allowlist youtube.com and youtu.be
- run containers as non-root
- mount filesystems with minimal permissions
- keep cookies and proxy credentials in secrets/environment configuration
- redact secrets from logs
- implement maximum execution time
- implement maximum file size
- implement concurrency limits
YT-DLP
Install current yt-dlp with: yt-dlp[default,curl-cffi]
Install:
- FFmpeg
- ffprobe
- Deno
Create configurable format policies:
- best
- 2160p
- 1440p
- 1080p
- 720p
- audio
PROXY SUPPORT
Create a ProxyProvider interface.
Environment configuration:
PROXY_ENABLED PROXY_URL PROXY_USERNAME PROXY_PASSWORD
Never log proxy credentials.
Proxy selection must happen per job/session rather than randomly changing an endpoint in the middle of a media transfer.
STORAGE
Use an S3-compatible interface.
Object layout:
youtube/{channel_id}/{video_id}/source.{ext} youtube/{channel_id}/{video_id}/metadata.json youtube/{channel_id}/{video_id}/thumbnail.jpg youtube/{channel_id}/{video_id}/subtitles/{language}.{ext}
DATABASE
Create tables for: downloads download_attempts media_objects
Downloads should include:
id platform source_url source_video_id channel_id status format_policy storage_key file_size checksum error_code created_at started_at completed_at
Add a uniqueness strategy for platform + source_video_id where appropriate.
OBSERVABILITY
Expose Prometheus metrics:
jobs_queued jobs_running jobs_completed_total jobs_failed_total download_duration_seconds download_bytes_total proxy_errors_total extractor_errors_total
ffmpeg_errors_total
storage_errors_total
LOGGING
Use JSON logs.
Every log line must contain: job_id worker_id stage
Never log: cookies authorization headers proxy passwords signed URLs
DOCKER
Create: Dockerfile.api Dockerfile.worker docker-compose.yml
Use health checks.
Pin dependency versions.
Create persistent volumes for PostgreSQL and Redis where appropriate.
Use ephemeral storage for worker downloads.
TESTING
Create: unit tests integration tests worker health checks
yt-dlp simulation tests
FFmpeg tests storage tests proxy connectivity tests
DOCUMENTATION
Create a detailed README containing:
installation environment variables architecture development deployment upgrades rollback monitoring troubleshooting
Generate all source code, Docker files, database migrations, tests and documentation.
Do not leave TODO placeholders.
Do not put real credentials anywhere in the repository.
AI Prompt: Audit an Existing Deployment
Once the coding agent has created the infrastructure, give it a second independent task:
Act as a senior SRE and security engineer.
Audit this entire yt-dlp deployment.
Do not rewrite the project yet.
Inspect:
- Dockerfiles
- Docker Compose
- Python dependencies
- yt-dlp version
- FFmpeg installation
- Deno installation
- queue implementation
- retry strategy
- proxy implementation
- storage implementation
- temporary file cleanup
- database transactions
- command injection risks
- SSRF risks
- secret leakage
- log leakage
- cookie handling
- resource exhaustion
- disk exhaustion
- concurrency bugs
- race conditions
- duplicate downloads
- graceful shutdown
- worker crash recovery
- monitoring gaps
- dependency pinning
- backup requirements
- rollback procedure
Classify findings:
CRITICAL HIGH MEDIUM LOW
For each finding provide:
- affected file
- affected code
- reason
- exploitation/failure scenario
- recommended fix
Then produce a prioritized remediation plan.
Do not claim an issue exists unless you can identify the relevant code path.
This second pass is valuable because coding agents are often better at finding mistakes when explicitly placed into review mode rather than being asked to review their own work while generating it.
Decide between self-hosting and VideoScale
The Operational Cost People Underestimate
The command:
yt-dlp URL
is free.
The infrastructure around it is not.
A self-hosted production system requires somebody to own:
servers containers queues databases storage FFmpeg JavaScript runtimes proxy infrastructure updates extractor regressions security patches monitoring retry logic disk cleanup autoscaling deployment incident response
This distinction becomes important when comparing self-hosted yt-dlp with a managed solution.
Self-Hosted yt-dlp vs VideoScale
For developers who only need occasional downloads, installing yt-dlp directly is often perfectly reasonable.
The equation changes when video downloading becomes part of a product.
With self-hosting, your application becomes responsible for an entire extraction infrastructure.
Conceptually:
Your Application
|
v
Queue
|
v
yt-dlp Workers
|
v
Proxy Infrastructure
|
v
FFmpeg
|
v
Storage
With a managed downloading layer such as VideoScale, the application-facing architecture can instead become:
Your Application
|
v
VideoScale
|
v
Video Result
The value is not that yt-dlp itself is difficult to execute.
The value is eliminating the operational machinery surrounding reliable downloads.
Why Use VideoScale Instead of Maintaining yt-dlp Yourself?
1. No yt-dlp Installation
You do not need to install or maintain:
Python yt-dlp yt-dlp-ejs Deno FFmpeg curl_cffi
on every application server.
Your application interacts with the managed service instead.
2. No Worker Fleet
Self-hosted systems eventually need:
worker containers queues autoscaling resource limits job timeouts dead-letter handling health checks rolling deployments
VideoScale moves that infrastructure outside your application stack.
3. No yt-dlp Update Pipeline
A production self-hosted environment should continuously track upstream yt-dlp changes.
The 2026 release history demonstrates why: JavaScript-runtime requirements changed, Python recommendations changed, security fixes landed, YouTube clients changed, and extractor behavior continues to evolve.
With a managed service, those compatibility updates become part of the service layer rather than your application deployment process.
4. Proxy Infrastructure Is Already Part of the Download Layer
At scale, outbound networking can become one of the most operationally demanding components.
VideoScale's download infrastructure uses rotating PrivateProxy residential proxy capacity as part of its networking layer, including session/rotation management and operational warming/health handling rather than requiring every customer to build their own proxy scheduler.
That means application developers do not need to maintain:
proxy pools credential distribution proxy health tracking session assignment rotation policies cooldowns regional routing logic
inside their own application.
5. Fewer Moving Parts
A self-hosted stack can contain:
API Redis PostgreSQL workers Docker yt-dlp FFmpeg Deno proxy provider object storage monitoring
Every component can fail independently.
Managed downloading reduces the number of components your own engineering team has to operate.
6. Easier Horizontal Scaling
Self-hosted scaling usually means deciding:
How many workers?
How many concurrent jobs per worker?
How much temporary disk?
How many proxy sessions?
How many fragment downloads?
When should workers autoscale?
What happens to jobs during shutdown?
A managed API makes the application side substantially simpler.
7. Cleaner Application Architecture
Instead of embedding video extraction infrastructure into your product, keep a clear boundary:
Product Logic
|
v
Video Download API
|
v
Storage / Processing
This separation can make the application easier to deploy, test and maintain.
When Should You Still Self-Host yt-dlp?
Self-hosting remains an excellent choice when:
- download volume is low;
- you need unusual yt-dlp options;
- you want complete infrastructure control;
- you have existing DevOps capacity;
- the workload runs internally;
- you need custom extractors or plugins;
- you need unusual FFmpeg processing;
- you already operate suitable storage and worker infrastructure.
For a developer downloading several videos occasionally, a single Linux box may be all that is necessary.
For a SaaS platform processing large numbers of user-submitted jobs, the operational equation is different.
Recommended Small Deployment
For a small workload:
1 Ubuntu server 1 yt-dlp installation FFmpeg Deno local SSD optional object storage
Keep it simple.
Do not introduce Kubernetes because you downloaded ten videos yesterday.
Recommended Medium Deployment
At moderate volume:
API Redis PostgreSQL 3–10 Docker workers S3-compatible storage proxy integration where legitimately needed Prometheus centralized logs
This gives you useful horizontal scalability without excessive complexity.
Recommended Large Deployment
At larger scale:
Load Balancer API replicas Managed PostgreSQL Managed Redis/SQS Container orchestrator Autoscaled worker pools Proxy management Object storage Metrics Distributed logs Alerts Dead-letter queues Canary yt-dlp deployments
At this point, you should explicitly compare the engineering cost of maintaining the stack against using VideoScale.
Validate production readiness
Production Checklist
Before calling your self-hosted downloader production-ready, verify all of the following.
Runtime
- [ ] Python 3.11+
- [ ] current yt-dlp build
- [ ] FFmpeg installed
- [ ] ffprobe installed
- [ ] supported JavaScript runtime installed
- [ ] yt-dlp-ejs support available
- [ ] curl_cffi available if impersonation is needed
Infrastructure
- [ ] queue configured
- [ ] retries configured
- [ ] dead-letter handling configured
- [ ] concurrency capped
- [ ] temporary disk monitored
- [ ] temporary files cleaned
- [ ] object storage configured
- [ ] database deduplication configured
Networking
- [ ] outbound connectivity tested
- [ ] proxy connectivity tested if applicable
- [ ] proxy credentials stored securely
- [ ] proxy sessions assigned consistently
- [ ] timeouts configured
- [ ] retry storms prevented
Security
- [ ] URLs validated
- [ ] no shell=True with user input
- [ ] file:// disabled
- [ ] containers run as non-root
- [ ] cookies stored as secrets
- [ ] proxy passwords stored as secrets
- [ ] logs redact credentials
- [ ] workers have minimal IAM permissions
- [ ] execution time capped
- [ ] resource consumption capped
Operations
- [ ] health endpoint
- [ ] readiness endpoint
- [ ] metrics
- [ ] structured logs
- [ ] alerts
- [ ] yt-dlp canary deployment
- [ ] rollback procedure
- [ ] dependency update procedure
Troubleshooting
"Requested format is not available"
First inspect formats:
yt-dlp -F "VIDEO_URL"
Then modify the format selector.
Avoid assuming that every video has exactly the same codecs and resolutions.
FFmpeg Errors
Check:
which ffmpeg
which ffprobe
Then:
ffmpeg -version
ffprobe -version
Remember that yt-dlp needs the actual FFmpeg binaries, not a Python package named "ffmpeg".
Extraction Suddenly Stops Working
Check your version:
yt-dlp --version
Then inspect current upstream releases.
Because websites change independently of yt-dlp, an older stable release can stop extracting correctly even though nothing changed on your server.
Test a current build in staging or canary before rolling it across production.
Proxy Fails
First isolate the proxy from yt-dlp:
curl \
--proxy "$YTDLP_PROXY" \
https://example.com/
Then:
yt-dlp \
--proxy "$YTDLP_PROXY"
--simulate
--verbose
"VIDEO_URL"
Determine whether the problem is:
DNS proxy authentication proxy connectivity TLS target extraction session consistency timeout
instead of repeatedly retrying the same command.
Worker Disk Is Full
Check:
df -h
Inspect temporary directories:
du -sh /tmp/*
Production workers should automatically clean failed jobs and enforce disk-space thresholds before accepting new jobs.
Final Architecture Recommendation
yt-dlp remains one of the most powerful building blocks available for developers working with online video in 2026.
For a single machine, the setup can be beautifully simple:
yt-dlp + FFmpeg + Deno
For production workloads, however, the architecture quickly evolves into:
API + Queue + Workers + yt-dlp + JavaScript runtime + FFmpeg + Proxy management + Temporary storage + Object storage + Database + Monitoring + Update pipeline
There is nothing inherently wrong with operating this infrastructure yourself. In fact, for teams requiring complete control, self-hosting can be the right decision.
But the comparison should be made against the whole system, not against the cost of running one command.
If your actual product requirement is simply:
«"Give my application a reliable way to retrieve a video without making video-extraction infrastructure part of my core stack,"»
then a managed service such as VideoScale can be substantially simpler.
Your engineering team can focus on what happens after the video is available—transcription, indexing, clipping, moderation, AI processing, analytics, archiving, or whatever your actual product does—rather than maintaining download workers, FFmpeg environments, proxy sessions, extraction compatibility, and yt-dlp updates.
And if you do decide to self-host, build the system around disposable workers, controlled concurrency, secure secrets, stable storage, observable queues, reproducible containers, and tested upgrades from day one.
That is the difference between running yt-dlp and operating yt-dlp at scale.