subscriptionfeed.git
109 min agofix channel search master
girst [Sun, 29 Nov 2020 02:24:13 +0000 (03:24 +0100)]
fix channel search

11 days agomake sure header+<video> fit above the fold
girst [Tue, 17 Nov 2020 13:50:47 +0000 (14:50 +0100)]
make sure header+<video> fit above the fold

4 weeks agodocument python version requirements
girst [Sat, 31 Oct 2020 12:24:03 +0000 (13:24 +0100)]
document python version requirements

4 weeks agoclean up broken-youtube-response handling
girst [Mon, 26 Oct 2020 22:04:48 +0000 (23:04 +0100)]
clean up broken-youtube-response handling

4 weeks agolink to sponsor.ajay.app; simplify js
girst [Mon, 26 Oct 2020 21:53:27 +0000 (22:53 +0100)]
link to sponsor.ajay.app; simplify js

now shows 0 segments when sponsorblock hasn't found the video id

4 weeks agofix search pagination
girst [Mon, 26 Oct 2020 21:49:19 +0000 (22:49 +0100)]
fix search pagination

old ?q= stopped displaying pages >= 2
protobuf field9 reversed by youtube-local

5 weeks agopolish some stuff
girst [Sat, 24 Oct 2020 14:14:42 +0000 (16:14 +0200)]
polish some stuff

5 weeks agofix /<something> 500 errors
girst [Thu, 22 Oct 2020 18:01:53 +0000 (20:01 +0200)]
fix /<something> 500 errors

m(

5 weeks agofurther improve <something> detector
girst [Tue, 20 Oct 2020 17:20:33 +0000 (19:20 +0200)]
further improve <something> detector

6 weeks agobetter /<something> algorithm
girst [Sat, 17 Oct 2020 23:13:41 +0000 (01:13 +0200)]
better /<something> algorithm

note: using strict_slashes=False instread of two route decorators to
avoid useless redirect.

7 weeks agoimplement a tiny port80-to-443 redirector
girst [Sat, 10 Oct 2020 15:59:54 +0000 (17:59 +0200)]
implement a tiny port80-to-443 redirector

7 weeks agofix reddit feed if 0 subscriptions
girst [Fri, 9 Oct 2020 13:09:47 +0000 (15:09 +0200)]
fix reddit feed if 0 subscriptions

7 weeks agoexclude search filters from new details style
girst [Thu, 8 Oct 2020 14:34:35 +0000 (16:34 +0200)]
exclude search filters from new details style

7 weeks agobetter <details> css
girst [Thu, 8 Oct 2020 14:22:11 +0000 (16:22 +0200)]
better <details> css

7 weeks agoupdate TODOs
girst [Sun, 4 Oct 2020 14:07:16 +0000 (16:07 +0200)]
update TODOs

7 weeks agoimplement pagination of channel/ucid/playlists (VERY HACKY)
girst [Sun, 4 Oct 2020 13:35:17 +0000 (15:35 +0200)]
implement pagination of channel/ucid/playlists (VERY HACKY)

i'm less ashamed of this code as i should be.

7 weeks agofix JSONDecodeError on search for real this time
girst [Sun, 4 Oct 2020 13:30:57 +0000 (15:30 +0200)]
fix JSONDecodeError on search for real this time

example of a broken response: https://paste.ubuntu.com/p/NbpSMWKQcX/

8 weeks agoprevent iter_content from raising ChunkedEncodingError when yt aborts connection
girst [Fri, 2 Oct 2020 09:52:39 +0000 (11:52 +0200)]
prevent iter_content from raising ChunkedEncodingError when yt aborts connection

2 months agoadd &t= to redirect-to-watch
girst [Mon, 28 Sep 2020 22:13:32 +0000 (00:13 +0200)]
add &t= to redirect-to-watch

2 months agosupport "?t=1h2m3s" and "?t=123s" style start-offsets
girst [Fri, 25 Sep 2020 14:41:08 +0000 (16:41 +0200)]
support "?t=1h2m3s" and "?t=123s" style start-offsets

documentation on media fragments:
https://www.w3.org/TR/2012/REC-media-frags-20120925/

2 months agoremove more button from xmlfeeds
girst [Thu, 24 Sep 2020 18:11:09 +0000 (20:11 +0200)]
remove more button from xmlfeeds

makes it necessary to remove the fallback from invidious (untested)

2 months agoremove v1 usage from /channels/
girst [Thu, 24 Sep 2020 16:49:51 +0000 (18:49 +0200)]
remove v1 usage from /channels/

doesn't appear to be used any more after a small and informal survey on

2 months agoadd sponsor segments debug info
girst [Thu, 24 Sep 2020 17:47:18 +0000 (19:47 +0200)]
add sponsor segments debug info

2 months agomake title in video cards selectable
girst [Wed, 23 Sep 2020 21:02:51 +0000 (23:02 +0200)]
make title in video cards selectable

2 months agofix exception on bad search results
girst [Wed, 23 Sep 2020 20:48:03 +0000 (22:48 +0200)]
fix exception on bad search results

r.json() can raise 'json.decoder.JSONDecodeError: Expecting value: line
1 column 1 (char 0)' when the response is an empty string

2 months agoremove superfluous title text
girst [Tue, 22 Sep 2020 12:11:22 +0000 (14:11 +0200)]
remove superfluous title text

2 months agovideo age: use 'm' for months, 'min' for minutes
girst [Mon, 21 Sep 2020 09:30:33 +0000 (11:30 +0200)]
video age: use 'm' for months, 'min' for minutes

2 months agochange pinned style
girst [Sat, 19 Sep 2020 23:15:18 +0000 (01:15 +0200)]
change pinned style

fully css defined; no more html boilerplate. also, the <hr> breaking
the grid was ugly.

2 months agoadd display flags to /c/*; dedup flag retrieval code
girst [Sat, 19 Sep 2020 23:11:04 +0000 (01:11 +0200)]
add display flags to /c/*; dedup flag retrieval code

2 months agoremove 'return "foo", 400' style error responses (browse/)
girst [Sat, 19 Sep 2020 18:54:15 +0000 (20:54 +0200)]
remove 'return "foo", 400' style error responses (browse/)

2 months agoexpose hls/dash streams to template
girst [Sat, 19 Sep 2020 18:12:58 +0000 (20:12 +0200)]
expose hls/dash streams to template

note that this can't be used, since no browsers implement native
hls/dash, and any js-polyfills will trigger cors-errors.

2 months ago(re)add subscribe button to channel page
girst [Sat, 19 Sep 2020 18:11:04 +0000 (20:11 +0200)]
(re)add subscribe button to channel page

2 months agoredirect youtube.com/<username>, fix /c/ trailing slash
girst [Sat, 19 Sep 2020 18:05:22 +0000 (20:05 +0200)]
redirect youtube.com/<username>, fix /c/ trailing slash

2 months agorename 'dangerous' blueprint, as its safe
girst [Sat, 19 Sep 2020 15:06:04 +0000 (17:06 +0200)]
rename 'dangerous' blueprint, as its safe

has been running long enough in production now that I'm relatively
confident that it doesn't get blocked by Big Goog' (*knocks on wood*).

2 months agoclean up dead protobuf structures
girst [Sat, 19 Sep 2020 14:57:06 +0000 (16:57 +0200)]
clean up dead protobuf structures

2 months agofix exception in pagination code on &foo=<emptystring>
girst [Sat, 19 Sep 2020 13:56:52 +0000 (15:56 +0200)]
fix exception in pagination code on &foo=<emptystring>

2 months agofix strange exception on reddit api usage
girst [Sat, 19 Sep 2020 13:53:19 +0000 (15:53 +0200)]
fix strange exception on reddit api usage

data.data might be null (maybe if subreddit list is []?)

2 months agodocument perl installation
girst [Sat, 19 Sep 2020 13:50:13 +0000 (15:50 +0200)]
document perl installation

2 months agoredirect /c/<ucid> to /channel/<ucid>
girst [Sat, 19 Sep 2020 13:49:09 +0000 (15:49 +0200)]
redirect /c/<ucid> to /channel/<ucid>

2 months agomove to v3 channel format
girst [Sat, 19 Sep 2020 13:47:48 +0000 (15:47 +0200)]
move to v3 channel format

modeled after
https://github.com/user234683/youtube-local/commit/1ff97bfde1467f7f18d4d2715a9357c41d9e9b8f

2 months agostore length when pinning videos
girst [Sat, 19 Sep 2020 13:25:55 +0000 (15:25 +0200)]
store length when pinning videos

2 months agoadd link to source code
girst [Thu, 17 Sep 2020 11:03:47 +0000 (13:03 +0200)]
add link to source code

2 months agofix pinning the first (oldest) subscription
girst [Thu, 17 Sep 2020 11:03:26 +0000 (13:03 +0200)]
fix pinning the first (oldest) subscription

2 months agoUse new k-anonymity SponsorBlock API; enable by default
girst [Wed, 16 Sep 2020 14:24:56 +0000 (16:24 +0200)]
Use new k-anonymity SponsorBlock API; enable by default

2 months agoimplement /watch video start offset
girst [Wed, 16 Sep 2020 13:42:46 +0000 (15:42 +0200)]
implement /watch video start offset

2 months agouse correct magic for v2 format
girst [Sat, 5 Sep 2020 16:31:46 +0000 (18:31 +0200)]
use correct magic for v2 format

test channel: UC2C_jShtL725hvbm1arSV9w (sort by popular, page2)

reversing of magic:
https://github.com/user234683/youtube-local/blob/master/youtube/channel.py#L47

2 months agolink to snopyta's invidous instance
girst [Fri, 4 Sep 2020 18:27:51 +0000 (20:27 +0200)]
link to snopyta's invidous instance

3 months agofix error on playlist that doesn't even work on youtube.com
girst [Sat, 29 Aug 2020 17:35:55 +0000 (19:35 +0200)]
fix error on playlist that doesn't even work on youtube.com

3 months agoI AM SO SMART -- S-M-R-T!
girst [Sat, 29 Aug 2020 16:22:23 +0000 (18:22 +0200)]
I AM SO SMART -- S-M-R-T!

3 months agofixup 'blockable'
girst [Sat, 29 Aug 2020 10:03:58 +0000 (12:03 +0200)]
fixup 'blockable'

3 months agofix aspect clamping yet again
girst [Sat, 29 Aug 2020 09:56:16 +0000 (11:56 +0200)]
fix aspect clamping yet again

3 months agouse new card.live property in typed_card
girst [Sat, 29 Aug 2020 09:52:07 +0000 (11:52 +0200)]
use new card.live property in typed_card

3 months agofix unavailable endcard
girst [Sat, 29 Aug 2020 09:43:46 +0000 (11:43 +0200)]
fix unavailable endcard

test id: WYjgHLFZMa0

3 months agocomment UU* (user uploads) playlists
girst [Sat, 29 Aug 2020 09:23:41 +0000 (11:23 +0200)]
comment UU* (user uploads) playlists

3 months agomore lenient channel redirection, catch not existing channels
girst [Sat, 29 Aug 2020 09:18:59 +0000 (11:18 +0200)]
more lenient channel redirection, catch not existing channels

3 months agowe're blockable :(
girst [Sat, 29 Aug 2020 09:16:38 +0000 (11:16 +0200)]
we're blockable :(

3 months agoconvert all search result types to use pipe framework
girst [Sat, 29 Aug 2020 09:14:05 +0000 (11:14 +0200)]
convert all search result types to use pipe framework

3 months agomake |G() take multiple keys
girst [Sat, 29 Aug 2020 08:30:58 +0000 (10:30 +0200)]
make |G() take multiple keys

3 months agoimplement 'pipe' framework and test on VIDEO searchresults
girst [Sat, 29 Aug 2020 08:20:34 +0000 (10:20 +0200)]
implement 'pipe' framework and test on VIDEO searchresults

3 months agonew searchresult types, 'no more results' variant
girst [Sat, 29 Aug 2020 07:10:16 +0000 (09:10 +0200)]
new searchresult types, 'no more results' variant

3 months agoadd /results as alias for /search
girst [Wed, 19 Aug 2020 21:51:42 +0000 (23:51 +0200)]
add /results as alias for /search

youtube-url compat

3 months agoadd final / to /channel/ urls in templates
girst [Wed, 19 Aug 2020 17:22:26 +0000 (19:22 +0200)]
add final / to /channel/ urls in templates

TECHDEBT: this should just use url_for!

3 months agomore innertube renderers
girst [Wed, 19 Aug 2020 17:14:35 +0000 (19:14 +0200)]
more innertube renderers

3 months agouse getpass() instead of input()
girst [Wed, 19 Aug 2020 15:53:23 +0000 (17:53 +0200)]
use getpass() instead of input()

input() writes prompt to stdout, so using it in scripts breaks. (WTF)

3 months agoimplement password change
girst [Wed, 19 Aug 2020 15:32:10 +0000 (17:32 +0200)]
implement password change

3 months agofix learning playlists
girst [Tue, 18 Aug 2020 20:05:43 +0000 (22:05 +0200)]
fix learning playlists

3 months agodon't need to patch youtubedown any more
girst [Mon, 17 Aug 2020 19:24:01 +0000 (21:24 +0200)]
don't need to patch youtubedown any more

as of somewhere between Aug 15 00:52 (last unpatched version downloaded)
and Mon, 17 Aug 2020 19:07:08 GMT (HTTP Last-Modified), it is no longer
necessary to modify youtubedown, as jwz has accepted my patch regarding
caller(). The %ciphers hash has an entry of 13 Aug 2020, so that's
probably the date he applied it locally.

3 months agodon't rely on calc(max()) css to get video to display
girst [Mon, 17 Aug 2020 19:08:39 +0000 (21:08 +0200)]
don't rely on calc(max()) css to get video to display

couldn't watch on my firefox-android 68 (but can now :D)

3 months agoraise exception on webhooks errors
girst [Mon, 17 Aug 2020 16:51:17 +0000 (18:51 +0200)]
raise exception on webhooks errors

logs the traceback.

3 months agoclean up v1/v2 channel protobuf handling
girst [Mon, 17 Aug 2020 16:50:21 +0000 (18:50 +0200)]
clean up v1/v2 channel protobuf handling

3 months agofix channel-playlists
girst [Mon, 17 Aug 2020 16:49:59 +0000 (18:49 +0200)]
fix channel-playlists

has no page number

3 months agofix "learning playlist" and playlists in general, more search result types
girst [Mon, 17 Aug 2020 16:48:15 +0000 (18:48 +0200)]
fix "learning playlist" and playlists in general, more search result types

3 months agohandle absent streamingData.formats/adaptiveFormats
girst [Mon, 17 Aug 2020 15:22:20 +0000 (17:22 +0200)]
handle absent streamingData.formats/adaptiveFormats

e.g. on (announced) livestreams/premiere videos

3 months agopaper over exception on dangerous.search
girst [Mon, 17 Aug 2020 14:52:40 +0000 (16:52 +0200)]
paper over exception on dangerous.search

wasn't able to reproduce this.

[2020-08-17 16:47:53,157] ERROR in app: Exception on /search [GET]
Traceback (most recent call last):
  File "/opt/yt/venv/lib64/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/yt/venv/lib64/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/yt/venv/lib64/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/yt/venv/lib64/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/opt/yt/venv/lib64/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/yt/venv/lib64/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/yt/app/dangerous/__init__.py", line 31, in search
    results, extras = prepare_searchresults(yt_results)
  File "/opt/yt/app/common/innertube.py", line 26, in prepare_searchresults
    return parse_result_items(contents)
  File "/opt/yt/app/common/innertube.py", line 169, in parse_result_items
    r, e = parse_result_items(content['content']['verticalListRenderer']['items'])
  File "/opt/yt/app/common/innertube.py", line 136, in parse_result_items
    listget(content.get('viewCountText',{}).get('runs'),0,{}).get('text')), # "1,234 watching"
  File "/opt/yt/app/common/innertube.py", line 12, in listget
    return next(iter(obj[index:]), fallback)
TypeError: 'NoneType' object is not subscriptable

3 months agoignore more advertisements
girst [Sun, 16 Aug 2020 15:17:42 +0000 (17:17 +0200)]
ignore more advertisements

3 months agohacky way to get v1 and v2 channel responses (needs cleanup!)
girst [Sun, 16 Aug 2020 12:45:14 +0000 (14:45 +0200)]
hacky way to get v1 and v2 channel responses (needs cleanup!)

3 months agodon't crash cipher util when youtubedown can't decode cipher
girst [Sat, 15 Aug 2020 21:27:39 +0000 (23:27 +0200)]
don't crash cipher util when youtubedown can't decode cipher

3 months agofix dummy-card 1px offset
girst [Sat, 15 Aug 2020 20:38:19 +0000 (22:38 +0200)]
fix dummy-card 1px offset

old style removed the border, shifting the last row 1px to the right.

3 months agoREADME: add note regarding new channel protobuf
girst [Sat, 15 Aug 2020 20:14:20 +0000 (22:14 +0200)]
README: add note regarding new channel protobuf

3 months agofallback from dangerous/channel (must fix browse_ajax)
girst [Sat, 15 Aug 2020 19:57:00 +0000 (21:57 +0200)]
fallback from dangerous/channel (must fix browse_ajax)

3 months agomake xmlfeed catch all channel subpages
girst [Sat, 15 Aug 2020 19:54:35 +0000 (21:54 +0200)]
make xmlfeed catch all channel subpages

needed for proper invidious/invidition redirect and fallback_route.

3 months agomake csrf verification use http.parser instead of regexp
girst [Sat, 15 Aug 2020 19:18:13 +0000 (21:18 +0200)]
make csrf verification use http.parser instead of regexp

finally removes the unecessary csrf field from GET forms

3 months agodocument why we aren't deleting videos, clean up logging
girst [Sat, 15 Aug 2020 12:25:43 +0000 (14:25 +0200)]
document why we aren't deleting videos, clean up logging

3 months agosometimes videoDetails.lengthSeconds=="0"
girst [Fri, 14 Aug 2020 16:20:55 +0000 (18:20 +0200)]
sometimes videoDetails.lengthSeconds=="0"

microformat.playerMicroformatRenderer.lengthSeconds might still be good
then. seen on 'premiere' videos.

3 months agomake view count thousand seperator a bit larger
girst [Sun, 9 Aug 2020 10:21:40 +0000 (12:21 +0200)]
make view count thousand seperator a bit larger

3 months agofound new innertube edge case
girst [Sun, 9 Aug 2020 10:21:01 +0000 (12:21 +0200)]
found new innertube edge case

3 months agoRevert "secondary channel format"
girst [Sun, 9 Aug 2020 10:19:15 +0000 (12:19 +0200)]
Revert "secondary channel format"

useless if it doesn't work all the time

This reverts commit 4e3b56b967570b9498651cf758a5ca3c56640689.

3 months agosecondary channel format
girst [Sun, 9 Aug 2020 10:17:38 +0000 (12:17 +0200)]
secondary channel format

reversed by omarroth on #invidious. his message:
The channel you sent me uses a different format. YouTube uses:
```json
{
  "80226972:0:embedded": {
    "2:0:string": "UC6Om9kAkl32dWlDSNlDS9Iw",
    "3:1:base64": {
      "2:0:string": "videos",
      "3:1:varint": 3,
      "4:2:varint": 0,
      "6:3:varint": 1,
      "7:4:varint": 1,
      "61:5:base64": {
        "1:0:embedded": {
          "2:1:embedded": {
            "1:0:base64": {
              "1:0:varint": 120
            }
          }
        }
      }
    }
  }
}
```

where `"1:0:varint": 120` is the offset into the channel.

3 months agoshow LIVE badge on subscription feed
girst [Sat, 8 Aug 2020 21:37:18 +0000 (23:37 +0200)]
show LIVE badge on subscription feed

3 months agoadd livestream detection for subscriptions
girst [Sat, 8 Aug 2020 21:13:35 +0000 (23:13 +0200)]
add livestream detection for subscriptions

note that we usually fetch get_video_info so fast that even for
unannounced live streams, isLive will be false (with isUpcoming true).

3 months agomake version closer to youtube
girst [Sat, 8 Aug 2020 18:54:00 +0000 (20:54 +0200)]
make version closer to youtube

don't think it makes a difference compat-wise (invidious only has
"2.<hardcoded-date>")

3 months agoremove stray end tag
girst [Sat, 8 Aug 2020 16:45:21 +0000 (18:45 +0200)]
remove stray end tag

3 months agomove search css into a stylesheet
girst [Sat, 8 Aug 2020 16:40:02 +0000 (18:40 +0200)]
move search css into a stylesheet

3 months agofix card macro behavriour w.r.t empty channel_id
girst [Sat, 8 Aug 2020 16:17:39 +0000 (18:17 +0200)]
fix card macro behavriour w.r.t empty channel_id

3 months agofix writing youtubedown (unicode chars)
girst [Fri, 7 Aug 2020 19:40:25 +0000 (21:40 +0200)]
fix writing youtubedown (unicode chars)

the cron job runs in LANG=C, so python defaults to ASCII encoding. also
redownload youtubedown if the on-disk size if 0 (e.g. due to this bug)

3 months agocorrect statement about idempotency of cronjobs
girst [Fri, 7 Aug 2020 19:22:20 +0000 (21:22 +0200)]
correct statement about idempotency of cronjobs

3 months agodisplay length on /feed/subscriptions and use new infocard renderer
girst [Fri, 7 Aug 2020 19:21:28 +0000 (21:21 +0200)]
display length on /feed/subscriptions and use new infocard renderer

3 months agoremove broken logging statement
girst [Fri, 7 Aug 2020 19:20:44 +0000 (21:20 +0200)]
remove broken logging statement

3 months agofetch length for new subscription videos
girst [Fri, 7 Aug 2020 19:19:19 +0000 (21:19 +0200)]
fetch length for new subscription videos

in the future, we'll also detect whether it's a livestream

3 months agoimplement video badge (for length)
girst [Fri, 7 Aug 2020 16:23:53 +0000 (18:23 +0200)]
implement video badge (for length)

Imprint / Impressum