9.3 KiB
subsyt
description
subsyt is a wrapper around yt-dlp1 to download youtube channels
based on a OPML file containing all your subscriptions, sorting the
channels into {show}/{season} folders, and generates nfo files,
extracts thumbnails, downloads posters, banners, and fanart so the media
should plug into media libraries well-enough, e.g. Jellyfin and Kodi.
A quick rundown on how to use it:
- download
subsytor build it into a binary yourself2 - install
yt-dlp3 - patch it with POT support (POT optional -- yet recommended) 4
- generate and download a OPML file5
- setup a config file6
- run
subsyt7
install
go install git.meatbag.se/varl/subsyt@latest
yt-dlp
Install pipx on your system.
sudo apt install pipx # debian
sudo pacman -Syu python-pipx # archlinux
pipx install yt-dlp
running
Configuration can be loaded from a file specified either by the env
variable CONFIG or --config flag.
The --config flag has priority over CONFIG environment variable.
CONFIG="/path/to/config.json" ./subsyt
./subsyt --config="/patch/to/config"
./subsyt # assumes "./config.json"
build
We want a statically linked binary so disable CGO.
CGO_ENABLED=0 go build
config
Full config.json:
{
"daemon": true,
"dry_run": true,
"out_dir": "./vids",
"provider": {
"youtube": {
"verbose": false,
"cmd": "./yt-dlp",
"format": "best",
"format_sort": "res:1080",
"output_path_template": "s%(upload_date>%Y)s/%(channel)s.s%(upload_date>%Y)Se%(upload_date>%m%d)S.%(title)s.%(id)s.%(ext)s",
"url": "https://www.youtube.com",
"throttle": 5,
"cookies_file": "",
"opml_file": "./youtube_subs.opml",
"po_token": "",
"schedule": "",
"bgutil_server": "http://127.0.0.1:4416"
}
}
}
Minimal config.json:
{
"out_dir": "./vids",
"provider": {
"youtube": {
"cmd": "./yt-dlp",
"throttle": 5,
"opml_file": "./youtube_subs.opml"
}
}
}
generate opml
Use this javascript snippet: https://github.com/jeb5/YouTube-Subscriptions-RSS to generate a file that has the format:
<?xml version="1.0"?>
<opml version="1.1">
<body>
<outline ...>
<outline text="" title="" xmlUrl="" .../>
<outline text="" title="" xmlUrl="" .../>
<outline text="" title="" xmlUrl="" .../>
</outline>
<outline ...>
<outline text="" title="" xmlUrl="" .../>
<outline text="" title="" xmlUrl="" .../>
<outline text="" title="" xmlUrl="" .../>
</outline>
</body>
</opml>
cookies
Warning
Your account MAY be banned when using cookies ! Consider using a throw-away account.
Install an extension that can download cookies per site, e.g. for firefox: https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/
The steps for the browser is:
- install cookie export extension, allow in private mode
- open a private browsing session (e.g. incognito)
- go to youtube.com and login using a (throw-away) account
- export the cookies using extension, save to disk
- close private browsing session
- point
cookies_fileinconfig.jsonto the cookies-file
Cookies may need to be refreshed if/when they expire, if so, repeat steps 2-5.
You can also yt-dlp to do it for you, though that exports all the
cookies in the browser:
yt-dlp --cookies-from-browser {browser} --cookies cookies.txt
pot
Youtube has started requiring proof-of-origin tokens for some players, and it may help not getting hit with the "sign in to confirm you are not a bot" together with cookies.
Either add a manually generated POT to the config: po_token = "{TOKEN}" or, set up bgutils8 with the youtube extractor to do POT
generation automatically, in which case, leave the po_token as an
empty string ("").
# assumes pipx was used to install yt-dlp
pipx inject yt-dlp bgutil-ytdlp-pot-provider
Then change the provider.youtube option for cmd to the yt-dlp
binary in the modified venv, e.g. /home/varl/.local/bin/yt-dlp.
On the same machine, run the bgutils http server, e.g. with compose:
bgutil:
image: brainicism/bgutil-ytdlp-pot-provider
container_name: bgutil
restart: unless-stopped
ports:
- 4416:4416
If using default ports and it's available on localhost, yt-dlp will
pick up the plugin automatically and can be verified in the logs.
scheduling
systemd
Tip
Remember to change the
ExecStartpath to the venv'edyt-dlpbinary if using it.
~/.config/systemd/user/subsyt-archival.service
[Unit]
Description=subsyt archival of yt subscribtions
[Service]
Type=oneshot
ExecStart=/home/varl/yt/yt-dlp -U
ExecStart=/home/varl/yt/subsyt
WorkingDirectory=/home/varl/yt
~/.config/systemd/user/subsyt-archival.timer
[Unit]
Description=subsyt archival on boot and daily
[Timer]
OnCalendar=*-*-* 4:00:00
Persistent=true
AccuracySec=1us
RandomizedDelaySec=30
[Install]
WantedBy=timers.target
container
podman run --rm \
--volume=path/to/opml:/data/opml.xml \
--volume=path/to/vids:/data/vids \
registry.meatbag.se/varl/subsyt
compose
Runs in scheduled mode (0400 hours daily), with automatic POT generation and the bgutil-ytdlp-pot-provider server.
services:
subsyt:
image: registry.meatbag.se/varl/subsyt:latest
container_name: subsyt
user: 1000:1000
volumes:
- /opt/subsyt/youtube_subs.opml:/data/opml.xml
- /media/videos/youtube:/data/vids
bgutil:
image: brainicism/bgutil-ytdlp-pot-provider
container_name: bgutil
restart: unless-stopped
ports:
- 4416:4416
http api
Enable the built-in intake server to queue ad-hoc videos without editing the OPML export.
{
"http_api": {
"enable": true,
"listen": "0.0.0.0:6901",
"auth_token_file": "/run/secrets/subsyt-token",
"queue_file": "./tmp/api-queue.json"
}
}
Submit new downloads with bearer authentication:
curl \
-H "Authorization: Bearer $(cat /run/secrets/subsyt-token)" \
-H "Content-Type: application/json" \
--data '{"url":"https://youtu.be/VIDEO","out_dir":"Channel"}' \
http://127.0.0.1:6901/v1/videos
Requests reuse the configured yt-dlp binary, honor existing throttling, and
persist through restarts when queue_file is provided.
Check the pending queue for debugging:
curl -H "Authorization: Bearer super-secret" http://127.0.0.1:6901/status
result
.
├── Technology Connextras
│ ├── archive.txt
│ ├── fanart.jpg
│ ├── poster.jpg
│ ├── s2024
│ │ ├── Technology_Connextras.s2024e0611.Connextras_dishwasher_follow_up_the_sequel.0Kp3bjm55xw-1080p-thumb.jpg
│ │ ├── Technology_Connextras.s2024e0611.Connextras_dishwasher_follow_up_the_sequel.0Kp3bjm55xw-1080p.nfo
│ │ ├── Technology_Connextras.s2024e0611.Connextras_dishwasher_follow_up_the_sequel.0Kp3bjm55xw-1080p.webm
│ │ ├── Technology_Connextras.s2024e0712.Here_s_what_Numitron_tubes_in_an_actual_product_look_like.XgzL05Gojfw-1080p-thumb.jpg
│ │ ├── Technology_Connextras.s2024e0712.Here_s_what_Numitron_tubes_in_an_actual_product_look_like.XgzL05Gojfw-1080p.nfo
│ │ ├── Technology_Connextras.s2024e0712.Here_s_what_Numitron_tubes_in_an_actual_product_look_like.XgzL05Gojfw-1080p.webm
│ │ ├── Technology_Connextras.s2024e0909.Answering_your_pinball_questions_-_Williams_Aztec_Q_A.P3Y4d2aHnNE-1080p-thumb.jpg
│ │ ├── Technology_Connextras.s2024e0909.Answering_your_pinball_questions_-_Williams_Aztec_Q_A.P3Y4d2aHnNE-1080p.nfo
│ │ └── Technology_Connextras.s2024e0909.Answering_your_pinball_questions_-_Williams_Aztec_Q_A.P3Y4d2aHnNE-1080p.webm
│ ├── s2025
│ │ ├── Technology_Connextras.s2025e0330.Renewable_energy_means_we_can_stop_setting_money_on_fire_silly_billy.Y2qSaD1v4cQ-1080p-thumb.jpg
│ │ ├── Technology_Connextras.s2025e0330.Renewable_energy_means_we_can_stop_setting_money_on_fire_silly_billy.Y2qSaD1v4cQ-1080p.nfo
│ │ ├── Technology_Connextras.s2025e0330.Renewable_energy_means_we_can_stop_setting_money_on_fire_silly_billy.Y2qSaD1v4cQ-1080p.webm
│ │ ├── Technology_Connextras.s2025e0331.An_unplanned_trip_from_Chicago_to_Milwaukee_in_an_electric_car.3GUQdrpduo0-1080p-thumb.jpg
│ │ ├── Technology_Connextras.s2025e0331.An_unplanned_trip_from_Chicago_to_Milwaukee_in_an_electric_car.3GUQdrpduo0-1080p.nfo
│ │ └── Technology_Connextras.s2025e0331.An_unplanned_trip_from_Chicago_to_Milwaukee_in_an_electric_car.3GUQdrpduo0-1080p.webm
│ └── tvshow.nfo