varl/subsyt
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity
subsyt/README.md
Viktor Varland 3129508e05
All checks were successful
build / build (push) Successful in 1m57s
Details
docs: use space for hyperlink
2025-04-12 13:15:45 +02:00

252 lines
8.2 KiB
Markdown
Raw Blame History

# subsyt
## description
`subsyt` is a wrapper around `yt-dlp`[1] to download youtube channels
based on a OPML file containing all your subscriptions, sorting the
channels into `{show}/{season}` folders, and generates `nfo` files,
extracts thumbnails, downloads posters, banners, and fanart so the media
should plug into media libraries well-enough, e.g. Jellyfin and Kodi.
A quick rundown on how to use it:
- download `subsyt` or build it into a binary yourself[2]
- install `yt-dlp`[3]
- patch it with POT support (POT optional -- yet recommended) [4]
- generate and download a OPML file[5]
- setup a config file[6]
- run `subsyt`[7]
[1]: https://github.com/yt-dlp/yt-dlp
[2]: https://git.meatbag.se/varl/subsyt#install
[3]: https://git.meatbag.se/varl/subsyt#yt-dlp
[4]: https://git.meatbag.se/varl/subsyt#pot
[5]: https://git.meatbag.se/varl/subsyt#generate opml
[6]: https://git.meatbag.se/varl/subsyt#config
[7]: https://git.meatbag.se/varl/subsyt#running
## install
```
go install git.meatbag.se/varl/subsyt@latest
```
## yt-dlp
Install `pipx` on your system.
```
sudo apt install pipx # debian
sudo pacman -Syu python-pipx # archlinux
pipx install yt-dlp
```
## running
Configuration can be loaded from a file specified either by the env
variable `CONFIG` or `--config` flag.
The `--config` flag has priority over `CONFIG` environment variable.
```
CONFIG="/path/to/config.toml" ./subsyt
./subsyt --config="/patch/to/config"
./subsyt # assumes "./config.toml"
```
## build
We want a statically linked binary so disable CGO.
```sh
CGO_ENABLED=0 go build
```
## config
`config.toml`:
```toml
dry_run = true # set to `false` for real run
out_dir = "./vids" # path to archive vids
[provider]
[provider.youtube]
cmd = "./yt-dlp" # path to yt-dlp binary
quality = "res:1080" # set the preferred quality, blank for 1080
output_path_template = "s%(upload_date>%Y)s/%(channel)s.s%(upload_date>%Y)Se%(upload_date>%m%d)S.%(title)s.%(id)s-1080p.%(ext)s" # yt-dlp output template
url = "https://www.youtube.com" # full yt url
throttle = 5 # throttle yt request, 5s works well
range = "1:5:1" # downloads last 5 videos: [START][:STOP][:STEP]
after_date = "20250326" # only download videos after date
cookies_file = "" # pass user cookies to yt, blank to disable
opml_file = "./opml.xml" # the opml file to use
po_token = "" # manually pass a proof-of-origin token, blank to disable
verbose = true # debug info for provider
```
## generate opml
Use this javascript snippet:
https://github.com/jeb5/YouTube-Subscriptions-RSS to generate a file
that has the format:
```xml
<?xml version="1.0"?>
<opml version="1.1">
<body>
<outline ...>
<outline text="" title="" xmlUrl="" .../>
<outline text="" title="" xmlUrl="" .../>
<outline text="" title="" xmlUrl="" .../>
</outline>
<outline ...>
<outline text="" title="" xmlUrl="" .../>
<outline text="" title="" xmlUrl="" .../>
<outline text="" title="" xmlUrl="" .../>
</outline>
</body>
</opml>
```
## cookies
> [!WARNING]
> Your account **MAY** be banned when using cookies ! Consider using a
> throw-away account.
Install an extension that can download cookies per site, e.g. for
firefox: https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/
The steps for the browser is:
1. install cookie export extension, allow in private mode
1. open a private browsing session (e.g. incognito)
1. go to youtube.com and login using a (throw-away) account
1. export the cookies using extension, save to disk
1. close private browsing session
1. point `cookies_file` in `config.toml` to the cookies-file
Cookies may need to be refreshed if/when they expire, if so, repeat
steps 2-5.
You can also `yt-dlp` to do it for you, though that exports all the
cookies in the browser:
```
yt-dlp --cookies-from-browser {browser} --cookies cookies.txt
```
## pot
Youtube has started requiring proof-of-origin tokens for some players,
and it may help not getting hit with the "sign in to confirm you are not
a bot" together with cookies.
Either add a manually generated POT to the config: `po_token =
"{TOKEN}"` or, set up bgutils[2] with the youtube extractor to do POT
generation automatically.
```
# assumes pipx was used to install yt-dlp
pipx inject yt-dlp yt-dlp-get-pot
pipx inject yt-dlp bgutil-ytdlp-pot-provider
```
Then change the `provider.youtube` option for `cmd` to the `yt-dlp`
binary in the modified venv, e.g. `/home/varl/.local/bin/yt-dlp`.
On the same machine, run the bgutils http server, e.g. with compose:
```
bgutil:
image: brainicism/bgutil-ytdlp-pot-provider
container_name: bgutil
restart: unless-stopped
ports:
- 4416:4416
```
If using default ports and it's available on localhost, `yt-dlp` will
pick up the plugin automatically and can be verified in the logs.
[2]: https://github.com/Brainicism/bgutil-ytdlp-pot-provider
## scheduling
### systemd
> [!TIP]
> Remember to change the `ExecStart` path to the venv'ed `yt-dlp`
> binary if using it.
`~/.config/systemd/user/subsyt-archival.service`
```
[Unit]
Description=subsyt archival of yt subscribtions
[Service]
Type=oneshot
ExecStart=/home/varl/yt/yt-dlp -U
ExecStart=/home/varl/yt/subsyt
WorkingDirectory=/home/varl/yt
```
`~/.config/systemd/user/subsyt-archival.timer`
```
[Unit]
Description=subsyt archival on boot and daily
[Timer]
OnCalendar=*-*-* 4:00:00
Persistent=true
AccuracySec=1us
RandomizedDelaySec=30
[Install]
WantedBy=timers.target
```
## container
```
podman run --rm \
--env='CONFIG=/data/config.toml' \
--volume=path/to/config:/data/config.toml \
--volume=path/to/opml:/data/opml.xml \
--volume=path/to/cookies:/data/cookies.txt
registry.meatbag.se/varl/subsyt
```
## result
```
.
├── Technology Connextras
│   ├── archive.txt
│   ├── fanart.jpg
│   ├── poster.jpg
│   ├── s2024
│   │   ├── Technology_Connextras.s2024e0611.Connextras_dishwasher_follow_up_the_sequel.0Kp3bjm55xw-1080p-thumb.jpg
│   │   ├── Technology_Connextras.s2024e0611.Connextras_dishwasher_follow_up_the_sequel.0Kp3bjm55xw-1080p.nfo
│   │   ├── Technology_Connextras.s2024e0611.Connextras_dishwasher_follow_up_the_sequel.0Kp3bjm55xw-1080p.webm
│   │   ├── Technology_Connextras.s2024e0712.Here_s_what_Numitron_tubes_in_an_actual_product_look_like.XgzL05Gojfw-1080p-thumb.jpg
│   │   ├── Technology_Connextras.s2024e0712.Here_s_what_Numitron_tubes_in_an_actual_product_look_like.XgzL05Gojfw-1080p.nfo
│   │   ├── Technology_Connextras.s2024e0712.Here_s_what_Numitron_tubes_in_an_actual_product_look_like.XgzL05Gojfw-1080p.webm
│   │   ├── Technology_Connextras.s2024e0909.Answering_your_pinball_questions_-_Williams_Aztec_Q_A.P3Y4d2aHnNE-1080p-thumb.jpg
│   │   ├── Technology_Connextras.s2024e0909.Answering_your_pinball_questions_-_Williams_Aztec_Q_A.P3Y4d2aHnNE-1080p.nfo
│   │   └── Technology_Connextras.s2024e0909.Answering_your_pinball_questions_-_Williams_Aztec_Q_A.P3Y4d2aHnNE-1080p.webm
│   ├── s2025
│   │   ├── Technology_Connextras.s2025e0330.Renewable_energy_means_we_can_stop_setting_money_on_fire_silly_billy.Y2qSaD1v4cQ-1080p-thumb.jpg
│   │   ├── Technology_Connextras.s2025e0330.Renewable_energy_means_we_can_stop_setting_money_on_fire_silly_billy.Y2qSaD1v4cQ-1080p.nfo
│   │   ├── Technology_Connextras.s2025e0330.Renewable_energy_means_we_can_stop_setting_money_on_fire_silly_billy.Y2qSaD1v4cQ-1080p.webm
│   │   ├── Technology_Connextras.s2025e0331.An_unplanned_trip_from_Chicago_to_Milwaukee_in_an_electric_car.3GUQdrpduo0-1080p-thumb.jpg
│   │   ├── Technology_Connextras.s2025e0331.An_unplanned_trip_from_Chicago_to_Milwaukee_in_an_electric_car.3GUQdrpduo0-1080p.nfo
│   │   └── Technology_Connextras.s2025e0331.An_unplanned_trip_from_Chicago_to_Milwaukee_in_an_electric_car.3GUQdrpduo0-1080p.webm
│   └── tvshow.nfo
```
Reference in a new issue View git blame Copy permalink