# subsyt ## description `subsyt` is a wrapper around `yt-dlp`[1] to download youtube channels based on a OPML file containing all your subscriptions, sorting the channels into `{show}/{season}` folders, and generates `nfo` files, extracts thumbnails, downloads posters, banners, and fanart so the media should plug into media libraries well-enough, e.g. Jellyfin and Kodi. A quick rundown on how to use it: - download `subsyt` or build it into a binary yourself[2] - install `yt-dlp`[3] - patch it with POT support (POT optional -- yet recommended)[4] - generate and download a OPML file[5] - setup a config file[6] - run `subsyt`[7] [1]: https://github.com/yt-dlp/yt-dlp [2]: [install](#install) [3]: [ytdlp](#yt-dlp) [4]: [pot](#pot) [5]: [opml](#generate opml) [6]: [config](#config) [7]: [run](#running) ## install ``` go install git.meatbag.se/varl/subsyt@latest ``` ## yt-dlp Install `pipx` on your system. ``` sudo apt install pipx # debian sudo pacman -Syu python-pipx # archlinux pipx install yt-dlp ``` ## running Configuration can be loaded from a file specified either by the env variable `CONFIG` or `--config` flag. The `--config` flag has priority over `CONFIG` environment variable. ``` CONFIG="/path/to/config.toml" ./subsyt ./subsyt --config="/patch/to/config" ./subsyt # assumes "./config.toml" ``` ## build We want a statically linked binary so disable CGO. ```sh CGO_ENABLED=0 go build ``` ## config `config.toml`: ```toml dry_run = true # set to `false` for real run out_dir = "./vids" # path to archive vids [provider] [provider.youtube] cmd = "./yt-dlp" # path to yt-dlp binary quality = "res:1080" # set the preferred quality, blank for 1080 output_path_template = "s%(upload_date>%Y)s/%(channel)s.s%(upload_date>%Y)Se%(upload_date>%m%d)S.%(title)s.%(id)s-1080p.%(ext)s" # yt-dlp output template url = "https://www.youtube.com" # full yt url throttle = 5 # throttle yt request, 5s works well range = "1:5:1" # downloads last 5 videos: [START][:STOP][:STEP] after_date = "20250326" # only download videos after date cookies_file = "" # pass user cookies to yt, blank to disable opml_file = "./opml.xml" # the opml file to use po_token = "" # manually pass a proof-of-origin token, blank to disable verbose = true # debug info for provider ``` ## generate opml Use this javascript snippet: https://github.com/jeb5/YouTube-Subscriptions-RSS to generate a file that has the format: ```xml ``` ## cookies > [!WARNING] > Your account **MAY** be banned when using cookies ! Consider using a > throw-away account. Install an extension that can download cookies per site, e.g. for firefox: https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/ The steps for the browser is: 1. install cookie export extension, allow in private mode 1. open a private browsing session (e.g. incognito) 1. go to youtube.com and login using a (throw-away) account 1. export the cookies using extension, save to disk 1. close private browsing session 1. point `cookies_file` in `config.toml` to the cookies-file Cookies may need to be refreshed if/when they expire, if so, repeat steps 2-5. You can also `yt-dlp` to do it for you, though that exports all the cookies in the browser: ``` yt-dlp --cookies-from-browser {browser} --cookies cookies.txt ``` ## pot Youtube has started requiring proof-of-origin tokens for some players, and it may help not getting hit with the "sign in to confirm you are not a bot" together with cookies. Either add a manually generated POT to the config: `po_token = "{TOKEN}"` or, set up bgutils[2] with the youtube extractor to do POT generation automatically. ``` # assumes pipx was used to install yt-dlp pipx inject yt-dlp yt-dlp-get-pot pipx inject yt-dlp bgutil-ytdlp-pot-provider ``` Then change the `provider.youtube` option for `cmd` to the `yt-dlp` binary in the modified venv, e.g. `/home/varl/.local/bin/yt-dlp`. On the same machine, run the bgutils http server, e.g. with compose: ``` bgutil: image: brainicism/bgutil-ytdlp-pot-provider container_name: bgutil restart: unless-stopped ports: - 4416:4416 ``` If using default ports and it's available on localhost, `yt-dlp` will pick up the plugin automatically and can be verified in the logs. [2]: https://github.com/Brainicism/bgutil-ytdlp-pot-provider ## scheduling ### systemd > [!TIP] > Remember to change the `ExecStart` path to the venv'ed `yt-dlp` > binary if using it. `~/.config/systemd/user/subsyt-archival.service` ``` [Unit] Description=subsyt archival of yt subscribtions [Service] Type=oneshot ExecStart=/home/varl/yt/yt-dlp -U ExecStart=/home/varl/yt/subsyt WorkingDirectory=/home/varl/yt ``` `~/.config/systemd/user/subsyt-archival.timer` ``` [Unit] Description=subsyt archival on boot and daily [Timer] OnCalendar=*-*-* 4:00:00 Persistent=true AccuracySec=1us RandomizedDelaySec=30 [Install] WantedBy=timers.target ``` ## container ``` podman run --rm \ --env='CONFIG=/data/config.toml' \ --volume=path/to/config:/data/config.toml \ --volume=path/to/opml:/data/opml.xml \ --volume=path/to/cookies:/data/cookies.txt registry.meatbag.se/varl/subsyt ``` ## result ``` . ├── Technology Connextras │ ├── archive.txt │ ├── fanart.jpg │ ├── poster.jpg │ ├── s2024 │ │ ├── Technology_Connextras.s2024e0611.Connextras_dishwasher_follow_up_the_sequel.0Kp3bjm55xw-1080p-thumb.jpg │ │ ├── Technology_Connextras.s2024e0611.Connextras_dishwasher_follow_up_the_sequel.0Kp3bjm55xw-1080p.nfo │ │ ├── Technology_Connextras.s2024e0611.Connextras_dishwasher_follow_up_the_sequel.0Kp3bjm55xw-1080p.webm │ │ ├── Technology_Connextras.s2024e0712.Here_s_what_Numitron_tubes_in_an_actual_product_look_like.XgzL05Gojfw-1080p-thumb.jpg │ │ ├── Technology_Connextras.s2024e0712.Here_s_what_Numitron_tubes_in_an_actual_product_look_like.XgzL05Gojfw-1080p.nfo │ │ ├── Technology_Connextras.s2024e0712.Here_s_what_Numitron_tubes_in_an_actual_product_look_like.XgzL05Gojfw-1080p.webm │ │ ├── Technology_Connextras.s2024e0909.Answering_your_pinball_questions_-_Williams_Aztec_Q_A.P3Y4d2aHnNE-1080p-thumb.jpg │ │ ├── Technology_Connextras.s2024e0909.Answering_your_pinball_questions_-_Williams_Aztec_Q_A.P3Y4d2aHnNE-1080p.nfo │ │ └── Technology_Connextras.s2024e0909.Answering_your_pinball_questions_-_Williams_Aztec_Q_A.P3Y4d2aHnNE-1080p.webm │ ├── s2025 │ │ ├── Technology_Connextras.s2025e0330.Renewable_energy_means_we_can_stop_setting_money_on_fire_silly_billy.Y2qSaD1v4cQ-1080p-thumb.jpg │ │ ├── Technology_Connextras.s2025e0330.Renewable_energy_means_we_can_stop_setting_money_on_fire_silly_billy.Y2qSaD1v4cQ-1080p.nfo │ │ ├── Technology_Connextras.s2025e0330.Renewable_energy_means_we_can_stop_setting_money_on_fire_silly_billy.Y2qSaD1v4cQ-1080p.webm │ │ ├── Technology_Connextras.s2025e0331.An_unplanned_trip_from_Chicago_to_Milwaukee_in_an_electric_car.3GUQdrpduo0-1080p-thumb.jpg │ │ ├── Technology_Connextras.s2025e0331.An_unplanned_trip_from_Chicago_to_Milwaukee_in_an_electric_car.3GUQdrpduo0-1080p.nfo │ │ └── Technology_Connextras.s2025e0331.An_unplanned_trip_from_Chicago_to_Milwaukee_in_an_electric_car.3GUQdrpduo0-1080p.webm │ └── tvshow.nfo ```