README.md

   1 # Unnamed Youtube Frontend
   2
   3 **Contributors wanted!** Please send me an email (see commit log) or message `girst` on irc.freenode.net/#invidious
   4
   5 Test Instance: [https://subscriptions.gir.st/](https://subscriptions.gir.st/)
   6
   7 # Installation
   8
   9 see INSTALL file
  10
  11 # Features
  12
  13  - subscribe to channels and playlists
  14  - raw video/audio mode (downloadable)
  15  - automatically skips over sponsored video segments (javascript required)
  16  - proxies georestricted videos
  17  - subreddit viewer
  18  - extremely leight weight: requires << 100MB RAM or disk space (multi-user test instance runs on bottom-of-the-barrel 10$/y VPS)
  19
  20 # Non-Features
  21
  22  - no algorithmically determined video recommendations: Find additional content through Youtube search, endcards or Reddit
  23  - no youtube comments
  24
  25 # TODOs
  26
  27 If you can program in Python or know HTML/CSS, get in touch!
  28
  29 ## important/big features
  30
  31  - Find a recognizable/googleable/duckable name
  32    send suggestions!
  33  - 1080p support
  34    Problem: must mux video and audio streams client side; requiring javascript (non-MediaSource solution to avoid CORS)
  35  - /watch endpoint should allow continue playing playlist (&list=PL...&index=x)
  36  - support for autogenerated channels
  37    Problem: require different protobuf structures; cannot differentiate from normal channels
  38    Samples: UCbqBqmG-EXTUsp2v4SHGDRg, UCuVPpxrm2VAgpH3Ktln4HXg, UClgRkhTL3_hImCAmdLfDE4g
  39
  40 ## users' wishlist
  41
  42  - Tag javascripts with LibreJS
  43    Problem: SPDX not supported; LibreJS doesn't define a AGPLv3-only identifier
  44  - allow proxying all thumbnails/googlevideo URLs
  45    Note: should be disableable in the config; only touching proxy/ blueprint
  46
  47 ## internal stuff
  48
  49  - moar internal and public documentation
  50  - when `get_video_info` returns a 403ing video url, remove from cache and retry
  51    Problem: requires an additional HEAD to googlevideo.com; must check
  52  - immediately subscribe
  53    asynchronically call update-subs and pull-subs on subscribing, iff necessary
  54  - remove deleted videos from subscription feed
  55    Note: when a deletion websub comes in, check with `get_video_info`
  56    Note: for livestreams, check postlivedvr and queue deletion after it ended
  57  - make magic tokens revocable
  58  - cleanups:
  59    - split up common/common.py; move stuff into plugins
  60    - abstract database access
  61    - proper task queue for refesh-cipher, pull-subs, update-websub
  62    - fix all TODOs/XXXs
  63    - welcome message: maybe make dismissable, show on all pages?
  64
  65
  66 # Advanced Topics
  67
  68 ## Extending Unnamed Youtube Frontend
  69
  70 UYtF is trivially extensible using [Flask Blueprints]. Just drop a new
  71 blueprint in `app/` and register it in the `[frontend]modules` section of
  72 `config.ini`. The Blueprint inside a module is expected to be named `frontend`.
  73 An `example_blueprint` is provided, which shows off how to use `fallback_route`
  74 and fetching the user token on guest-accessible routes.
  75
  76 The default `youtube` blueprint provides unblockable but minimal versions of
  77 the /channel and /playlist endpoints. A custom blueprint (or, as an example,
  78 the `browse` blueprint) can overwrite existing routes by being listed before
  79 the one to be overridden in `config.ini`.
  80 A blueprint may also delegate a request to another (lower precedence) blueprint
  81 using `return fallback_route(*args, **kwargs)`. This can be useful if the
  82 endpoint might not be as reliable.
  83
  84 [Flask Blueprints]: https://flask.palletsprojects.com/en/1.1.x/blueprints/
  85
  86 ## Proxying videos
  87
  88 This is useful to watch videos that are IP-locked. add the `proxy` blueprint to
  89 the `[frontend]modules` list in `config.ini` to enable (disabled by default, as
  90 it is resource-intensive, blocks a gunicorn thread and hasn't been tested
  91 w.r.t. googlebanning).
  92
  93 ## Guest User
  94
  95 The guest user doesn't really exist; you can't log in as guest, and all guest
  96 views are read-only. To modify the shown subscription feed, you'll have to
  97 modify the database manually. To add a single channel, issuing
  98 `INSERT INTO subscriptions (user, channel_id) VALUES ("guest", "UC...")`. To
  99 load many at once, prepare a file (guest.csv) of channel ids
 100 that looks like this:
 101
 102     UCxxxxxxxxxxxxxxxxxxxxxxxx,guest,channel
 103
 104 Then from the `sqlite3` console, issue:
 105     .mode csv
 106     .import guest.csv subscriptions
 107
 108 ## Running modules on different Hosts
 109
 110 The `webhooks` and `proxy` modules support being ran standalone on a different
 111 machine. For this, remove them from `config.ini`'s `[frontend]modules` and
 112 start them using gunicorn by replacing `app:app` with (e.g.)
 113 `app.webhooks:app()` (the parenthesis are required, as those use the factory
 114 pattern to not uselessly instantiate the Flask app object if it wouldn't be
 115 needed). An example is given in `config/subscriptions-webhooks.service`.
 116
 117 Note that the proxy endpoint can't reliably access ip-restricted streams if its
 118 IP is different to the frontend's.
 119
 120 ## Functional Overview
 121
 122 [rough functional overview diagram](https://viewer.diagrams.net/?highlight=0000ff&edit=_blank&layers=1&nav=1&title=Copy%20of%20youtube-frontend-2020jul19.drawio#R7V1bk5s4Fv41rt19cBcXc3vsS9w7tZlKV7pqJr0vKRlkrAlGDsjddn79SoAwSDKmuw2Y3kmlKkgIgb9zPzpSJubtenefgM3qdxzAaGJowW5i3k0MQ58ZxoT91YJ93uM4s7wjTFBQDDp0PKJfsOjUit4tCmBaG0gwjgja1Dt9HMfQJ7U%2BkCT4pT5siaP6WzcghFLHow8iufdPFJBV3usazqH%2F3xCFK%2F5m3fbyO2vABxe%2FJF2BAL9UusxPE%2FM2wZjkV%2BvdLYwYeByX%2FLn5kbvlhyUwJm0eiKzH%2ByV2vj8tnrz%2Ffr%2F9%2FVEPl1PHzqd5BtG2%2BMXF15I9h4B%2B%2BIZd0lcRBKKvFGYQh%2BzuzcsKEfi4AT67%2F0I5gPatyDqiLZ1eLjAheF00kgIldh3BJb8kmD50p9GrJYqiWxzhhLZjHLP5E7wlKA5v6U%2BEyTfaP9WuLNofgHQFAz4xJoAgHLPbHptIRqYA6xkmBO4qXQVS9xCvIUn2dEhx17XyJwquNfSi%2FXLgAWtWEHZVob9hFp2g4LuwnPpAGnpRUEdNqbv%2F%2FNIf8cOXH0b6ZWk6f3yaJmiqm6cpReGKgwwX7TRxFHDXYD0DilyICxRtW5NQnNkKFE3tDCgq%2Bd1qwe%2BXjqIMYsl1VRBL%2Fjw7iHziCmYwoFqzaOKErHCIYxB9OvTe1FE9jPmMMwXAsPoLErIvTADYElxHOiVU%2F1wzlX6AOuubI%2Fb52bQwDvgIPwJpivy8sxjCpoE7RL7x4fT6iV1nSoW17naVW3f7onGUiCneJj5sgIrrDfqdISRNA%2FNxDMdGnkhgRJXdc906qSicPUrBAPvKgA1GMUkrMz%2BwjgqruXVeswo7d%2BCWfMYD75Sf9nZ24gJxEMkIUwPMnAZAwAKk8KhJYgNSgpMWtugcgmjV0TFdWRRV6szuShBNbwhBpAAm%2B2%2B52Fi8nYuRV7YPgpS19tXWA0wQBQAmReeZJPvtUmq2FFLrIoTU5C4mZ0NTcPyE8bZjNY3vRqhV3oodEWYe0TO9DNnl9cNvvJe%2BpHJDMdYH%2FgoeG31EPfj7CFFeT9p4qplQfF6UHcD%2FEWai8mVL6CzwfGrENAX6Da1GrNmI7fnbpX7W1jRfhthbptmpbd7vo%2Fvd0%2B1zGPz8uf41%2Fwzsr8FUl93jPi1M2ah4aU3Whft2V1TIav6d1%2BzgsYZold7h9CnsiRJd9yIYyxQCDMs2Gu2JON71erAnM4U9OaL1gU9wC5V%2FBk1ua3UkZp4jaXJVlqAzTW6PWZMfpHdWk129R9FtHa4ZFyG69syuMaCu9yCKjv4hmMytMZn7aiarhkE1M3XCSHXPnLNOeFN2SDxR%2BwlqLf%2FQ4qkDV76WyR2u6%2Fl7CofsaLxjaE3juxEKS7JPT3hLtgsasmjs1YZ2j3EYQRrypLL4RBHapBlH88glwtugFxtmOXW0dEOORnRPYcT0zvKLTntbv4I7ELLU%2F82mIp5FbymxRi9ImnqdTy3Nk5BUAdmdN2AOoagretEU%2FHd3lNkhr6XavYww0Tbq2QWjD%2B%2FctgbmNFdIRGrjTES2ZbWzrxa8S894Cn0t5e9u0WZFgW6b7rtPwGJxfPwIzEG5KMcFUbFw1685GDaXQwM5XbAHjuaMUkx5AmQkJmEmJGyMEwsG0njd68GEuANzp%2F2qKG7kjHkZy82GJpTZuM2Mqbte0%2FiOlqe1NuZts42iabpdpH5yNWKjpduC6GsDG602ZU%2BXCKSp1zNzliZXkPUL5Csy6BcFpJDNsTR9WCDNoQ1VzU7pJ%2BzU8CapbdbSPLdJep%2B4yAk9hdafU7h%2BtI5p5gmOCcV1xPbBNGY1aVStePUb1NgDS6P%2BMTIPPHc5Es%2FRFkqaxNp30XG0vKbxHTmORhsV8idcULextQ6hw1cYlzpnkYhjR6hUBAuvKIfqV6WMen3zoJkOyZVcM83eopjeoVH4dp1hkiQnlyt1R2CgI8uVZ0twOCNmq%2FezQdfk1YV1zLKqsify8t9Z0fbbTQAInL4UGn4EmlgUkdnQqlheWvj6%2BEg7rsMwgSFghV4irvSnkjowIEIh2x3mZ9vIaAcDBPkgui5urFEQ5OIGU%2FQLLLKpmGQUbgid17qZWHdsLiphaS5sZ8JcqM62dXnVWNcUoBtdge7ICa9DLcNDgnf70WMuLtWrQFcxeneYy2kun0aLssUoK0VOKAOQbvLtsEu0Y%2FbjPBvRBPWg8NRUO9HOsSdSjZqc07pA1ATDZCiUar%2BoyamNy0OtRImbc0Uiul%2FU5KqkGL7Qju0mwiCgFzEmaElVHNuLLNd4jU1FGk490%2BPJ%2BOvKHdCdEcCVCJDAJYVmxYDOVl7QJtsHznzJn1uYkg9ABEHjWgoqqPYKdUcF2SNbQuIzGrCUqUZ9%2BAx47RkFEH88MSh3Y1cpoFJEnVHAld0zToFNBPYwufqLwq79kwKdAJ%2BRws9Kb%2F41elqYwv7bUjqqtHB6pYUc9FHFk30zjTW%2FZzLwHcVLPHrsZ0JoyDeRDigHhoS9Il%2B6ujLYppj588blSmnCPqlt%2BlSaMCUJBOuJQT9aI6vtehEDFKUts69jo7qtOXWqu%2FJyT1kw3osb5sqB0jaFyT9SwQkYv%2BWRzmJQVBAosT%2BHxKmPoBl65dt1hMW2N621HaXNyVym23bLMBeT%2FhezlYQrj6Dql3C9nrTitCVNN7ujXl36JhRaWWatxvLkeKePnX6uqmZIsoiPECSZ%2B5nvc7phh6699qSGS0qC68J6pD30gqSr2hAlkaHM0bYC%2Fs1bzhQHXp0Bc0NIgiuyDXzJvwp5Z1vQXDnMVfmCnPOpf7CNmLjftvYjE5Q96a9AHMPjDqT0HGD1Q1mkF6GUpG8uBBidO%2BTVWcTRzat2JZVWV0ziqSKQIwoOrbODJqt4i%2FBmJyGWvZ%2FBAkYPFMTiaEN%2BluJNxG7clMfTcFmcGOYy%2B0OHZC%2B75llZTZWiLb7nbkUIO0nzOtPdcz%2BItSvk43iZnZtz5dM3UjYCBNB%2FWD%2Flt%2Flim6IYpul0DWI6yZrSdKqzsyzmBjteb76g%2FsZUN9yrTRyqFUb2cupSZEfznFGL6MKWm3LncDVpbMocwvvO7zEPsuY%2FjMes9jxlr0w57uxO2fv85UEqgNT%2BstOjw9xEwqH9ZTEatvRmf1kY72rn9ZeboGp2Gr7CIECk%2FblnEWKk%2FUj1fqKD7SjyuF052Eq6vWKPyd92fAA77vJyjaHsuBwHJ4UUjyqsclQnDXYUVrXShYw92ec1h67SY3xt0QZrBmO8SDeVgekGxHwkWw0uJ63eaP0u1azpdpHTP33l7COPwcpj0Dk%2FeXIE5nYUgan9a1kwA5RSVDJgDO0h2oboQ6wF1IH3VCufekdrAcqjC%2F%2BPApumoxtPHvHIidJ%2FZNP02c1a%2BY9ilbSo9fwwfqbb4wkcSvRVScW%2F%2FczL8TM9rT8%2Fs1FbVDgkzI6Nk%2BO%2FopRhVP6nqzBbXfmfanzlep1NruTKejWNrADLI9CfzRoIMgeVMHXIRIIVeNLL3x7G71MYwlZeV7HmoiyxfYNPQZuH%2F%2Bgoz7Yc%2Frso89P%2FAA%3D%3D)
 123
 124
 125 ## License
 126
 127 This project is licensed under the [GNU Affero General Public License, Version 3].
 128
 129 [GNU Affero General Public License, Version 3]: LICENSE