README.md

   1 # Unnamed Youtube Frontend
   2
   3 Note: this is a work in progress, still unreleased software. Feel free to look around, but keep in mind that this repository will move servers for its public release.
   4
   5 Contributors wanted! Please send me an email (see commit log) or contact `girst` on irc.freenode.net (hanging out in `#invidious` for now)
   6
   7 Test Instance: http://delta.gir.st:8000/ (will move in the future)
   8
   9 TODO:
  10  - task queue for refesh-cipher, pull-subs, update-websub
  11     - should handle delays itself
  12     - would allow us to just put a websub/pullsub request into the queue
  13       whenever a user subscribes to a channel, without worrying about
  14       ratelimiting ourselves.
  15  - webhook: as blueprint; main app looks in envvar/config what to load?
  16  - abstract database access
  17    we want to be able to choose between at least sqlite and postgres
  18  - implement 'dangerous' functions as disablable blueprint
  19    - search through invidious-api
  20    - proxy for subtitles, hls/dash manifests, ...
  21      - subtitles on /watch
  22      - hls.js for livestreams
  23    - proxy for geolocked videos?
  24  - modern css frontend (flexbox)
  25  - document all the things
  26  - quality of life improvements:
  27    - asynchronically call update-subs and pull-subs on subscribing, iff necessary
  28    - get_video_info on incoming websub-webhook to get exact published date
  29  - development-friendly error handling:
  30    - log unkown info/endcards to file
  31    - save get_video_info/reddit.json/etc with traceback
  32  - proper support for sponsorblock (cached db?)
  33  - allow pinning non-subscribed videos
  34    would make reddit easier, less-surprising when unsubscribing from channels with pinned videos
  35  - cleanups:
  36    - move subsystem-specific stuff out of common/common.py (e.g. reddit)
  37    - remove jsonify()
  38    - purge magic strings (e.g guest token)
  39    - fix all the TODOs and XXXs
  40    - clean up login stuff
  41      https://flask-login.readthedocs.io/en/latest/
  42      https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-v-user-logins
  43    - jinja2: for/else
  44    - welcome message: maybe make dismissable, show on all pages?
  45  - find a name!
  46  - /channel/UE.../videos 404s
  47  - invidious search: api returns 'nice' author strings (e.g. numberphile instead of UC...)
  48    if not channel_id.match(...): redirect('/user/...') (in invidious blueprint)
  49
  50 stuff to look at:
  51  - app.config.from_pyfile('the-config.cfg')
  52    NOTE: this will not work for the non-python/non-flask parts
  53
  54 wishlist:
  55  - proxy googlevideo (and probably thumnails) responses
  56
  57 # Installation
  58
  59 see INSTALL file
  60
  61 # Usage
  62
  63 Since user authentication is not yet implemented, users are for now identified by a randomized token. To "log in", just visit `/feed/subscriptions?token=....`. at the bottom, click 'Manage Subscriptions' and paste the UC...-URLs of channels you want to subscribe to.
  64
  65 If no token is specified, it defaults to `guest`. Since this is a readonly account, you can populate it by directly writing into the database:
  66
  67 first, prepare a file (guest.csv) of channel ids like so:
  68     UCxxxxxxxxxxxxxxxxxxxxxxxx,guest,channel
  69 then from the sqlite3 console, issue:
  70     .mode csv
  71     .import guest.csv subscriptions
  72
  73
  74 # Notes
  75  - caching external api requests
  76    currently using in-memory-backend, which is purged every 10minutes. a real installation should use redis, which should scale better and handle purging for us.
  77    the in-memory cache conflicts with gunicorn's worker model: which is a sepreate process for each request. switch to mulitple gthreads on a single worker to avoid getting the cache torn down after every request (note that threads may leak memory, if my observations from $dayjob are still valid).
  78  - neater sqlite output:
  79     .mode column
  80     .headers on
  81
  82 # Advanced Topics
  83
  84 ## Extending Unnamed Youtube Frontend
  85
  86 UYtF is trivially extensible using [Flask Blueprints]. Just drop a new
  87 blueprint in `app/` and register it in the `[frontend]modules` section of
  88 `config.ini`. The Blueprint inside a module is expected to be named `frontend`.
  89
  90 The default `youtube` blueprint provides unblockable but minimal versions of
  91 the /channel and /playlist endpoints. A custom blueprint (or, as an example,
  92 the `invidious` blueprint) can overwrite existing routes by being listed before
  93 the one to be overridden in `config.ini`.
  94 A blueprint may also delegate a request to another (lower precedence) blueprint
  95 using `return fallback_route(*args, **kwargs)`. This can be useful if the
  96 endpoint might not be as reliable (e.g. invidious-api being blocked)
  97
  98 [Flask Blueprints]: https://flask.palletsprojects.com/en/1.1.x/blueprints/