]>
git.gir.st - subscriptionfeed.git/log
girst [Sun, 23 Apr 2023 10:36:04 +0000 (10:36 +0000)]
reddit: handle youtube.com/live/<video_id> urls
girst [Sat, 22 Apr 2023 18:15:21 +0000 (18:15 +0000)]
browse: hack channel title back in
girst [Sat, 22 Apr 2023 18:02:38 +0000 (18:02 +0000)]
switch to yt-style protobuf for channel videos/livestreams/shorts
shorts only works in this format; page>=2 still broken
girst [Sat, 22 Apr 2023 18:01:41 +0000 (18:01 +0000)]
make first page of shorts work
google-provided continuation gives an error, even though our protobuf structure looks right
girst [Sat, 22 Apr 2023 17:21:50 +0000 (17:21 +0000)]
WIP: port channel videos+livestreams
girst [Sat, 22 Apr 2023 17:13:22 +0000 (17:13 +0000)]
browse: implement channel livesteams subpage
girst [Sat, 22 Apr 2023 16:26:23 +0000 (16:26 +0000)]
fix video playback
android parameters needed updating, most importantly adding androidSdkVersion and user agent.
girst [Sun, 19 Mar 2023 20:22:29 +0000 (20:22 +0000)]
support /live/ urls
girst [Sun, 19 Mar 2023 20:18:09 +0000 (20:18 +0000)]
browse/channels: prepare for livestreams, shorts
girst [Sun, 19 Mar 2023 20:18:09 +0000 (20:18 +0000)]
browse/channels: remove nonfunctional 'oldest' sorting
girst [Sun, 19 Mar 2023 20:18:09 +0000 (20:18 +0000)]
hackily implement @usernames
girst [Sun, 19 Mar 2023 20:18:09 +0000 (20:18 +0000)]
browse/channel: fix popular sorting
don't rely on the fallback to support most popular videos.
girst [Sun, 19 Mar 2023 20:18:09 +0000 (20:18 +0000)]
fix (un)pinning/(un)hiding
somehow, this code broke; possibly due to a flask change?
girst [Sun, 19 Mar 2023 20:18:09 +0000 (20:18 +0000)]
search: stub out "Latest Shorts from <username>" result groups
girst [Sun, 19 Mar 2023 20:18:09 +0000 (20:18 +0000)]
fix channel pages
youtube changed some stuff again.
see also: https://github.com/TeamNewPipe/NewPipeExtractor/pull/944
girst [Sun, 14 Aug 2022 22:24:29 +0000 (00:24 +0200)]
youtube: support /shorts urls
minimal version, just redirect to /watch
girst [Sun, 14 Aug 2022 22:23:22 +0000 (00:23 +0200)]
reddit: support shorts urls
girst [Fri, 12 Aug 2022 15:58:12 +0000 (17:58 +0200)]
fix pinning videos for python > 3.6
this worked on 3.6/rhel7, but doesn't on 3.9/rhel8
girst [Fri, 12 Aug 2022 15:53:43 +0000 (17:53 +0200)]
don't crash when encountering a new "View corrections" card
https://support.google.com/youtube/answer/57404?hl=en
https://www.theverge.com/2022/6/15/
23168884 /youtube-add-corrections-feature-annotations-creators?scrolla=
5eb6d68b7fedc32c19ef33b4
girst [Fri, 29 Apr 2022 18:27:51 +0000 (20:27 +0200)]
allow selecting muxed video quality
listing all urls in the <video> element doesn't work (in firefox at
least), so only load highest resolution one by default. we can then just
override that choice when a &itag=XX is present.
girst [Sun, 3 Apr 2022 15:46:36 +0000 (17:46 +0200)]
move inline styles to blueprint sheet
girst [Sun, 3 Apr 2022 15:45:28 +0000 (17:45 +0200)]
use simple links instead of text-only form submit buttons
this was done in the past to preserve some url parts, but isn't
necessary now.
girst [Wed, 30 Mar 2022 13:17:26 +0000 (15:17 +0200)]
fix "agegate bypass failed" error message
TVHTML5* playabilityStatus has only {status, reason}
girst [Tue, 29 Mar 2022 15:05:51 +0000 (17:05 +0200)]
clean up new agegate bypass
girst [Tue, 29 Mar 2022 14:55:53 +0000 (16:55 +0200)]
port yt-dlp#3233 agegate bypass
the change in embedUrl was cargo-culted from there, no idea if needed.
girst [Fri, 18 Feb 2022 09:24:49 +0000 (10:24 +0100)]
encard parser: videoDuration might be absent
possibly removed from every VIDEO card, not investigated.
girst [Mon, 14 Feb 2022 11:14:26 +0000 (12:14 +0100)]
fix reddit length parsing
- allow first group to be a single digit (e.g. 9:07 or 1:31:26)
- fix obviously wrong labels (e.g. 20:59:00 => 20:59) that occur
relatively often
girst [Sun, 13 Feb 2022 16:26:39 +0000 (17:26 +0100)]
remove fetch_xml call from browse/playlist
no need for this, since youtube now apparently returns this information
for us.
girst [Sun, 6 Feb 2022 19:50:34 +0000 (20:50 +0100)]
allow sorting by most popular for fallback route
girst [Sun, 6 Feb 2022 17:45:36 +0000 (18:45 +0100)]
make music mix playlists work
needs some cleanup
girst [Sun, 6 Feb 2022 15:34:58 +0000 (16:34 +0100)]
remove UC->UU playlist hack
doesn't work on youtube-proper; iirc just cargoculted from invidious
girst [Wed, 2 Feb 2022 16:13:50 +0000 (17:13 +0100)]
improve reddit video length matching
- strict xx:yy or xx:yy:zz format
- ignore hours if 00 (not 00 minutes)
girst [Sat, 8 Jan 2022 23:11:44 +0000 (00:11 +0100)]
remove 2nd no-url bailout again
this replaces
18bf5ede ; the problem was that without url, the call to
urlparse() would fail. bailing out at this point however breaks
livestreams (and premieres). the real fix is of course to not check for
geolocking if there's no url.
girst [Sat, 8 Jan 2022 22:54:59 +0000 (23:54 +0100)]
only show 'SOON' badge if video hasn't premiered yet
note: during the livestream phase, the final video length time will
already be shown.
girst [Sat, 8 Jan 2022 00:06:36 +0000 (01:06 +0100)]
finally good /c/ and /user/ matching
don't look up both types of channels and return just one, but do the
correct lookup in the first place. no point in doing the fallback lookup
now that i'm fairly certain the lookup does the right thing.
note: using @app.route('/<any(user, c):typ>/<user>/') doesn't work,
probably because it then doesn't override the hardcoded rule in
youtube/.
girst [Wed, 5 Jan 2022 22:05:00 +0000 (23:05 +0100)]
get_video_info: bail out if no stream url is found
this was triggered relatively often by the websub-webhook (who doesn't
need the url anyways).
girst [Wed, 5 Jan 2022 21:58:53 +0000 (22:58 +0100)]
implement displaying 'youtube mixes' in /channel/.../playlists
girst [Wed, 5 Jan 2022 21:45:58 +0000 (22:45 +0100)]
ignore another advert search result
for youtubekids.com, when searchign for youtube kids
girst [Wed, 5 Jan 2022 21:42:10 +0000 (22:42 +0100)]
ignore hashtag related search results
includingResultsForRenderer is shown when searching e.g. for '#foo' to
also show results for 'foo'. hashtagTileRenderer is a link to
'/hashtag/<search_query>' telling you that there are X videos from X
channels about that topic.
girst [Wed, 5 Jan 2022 21:35:19 +0000 (22:35 +0100)]
ignore useless info message search result
happens when searching for a youtube url, e.g.
/search?q=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dxxxxxxxxxxx
girst [Wed, 5 Jan 2022 21:33:46 +0000 (22:33 +0100)]
ignore another search result ad type
girst [Wed, 5 Jan 2022 21:25:56 +0000 (22:25 +0100)]
remove old commented out templating code
girst [Wed, 5 Jan 2022 21:09:24 +0000 (22:09 +0100)]
fix COURSE/"learning playlist" special-casing
this appears to no longer be needed. new test case (since the tomscott
playlist can't be searched for any more):
- PLRqwX-V7Uu6Zy51Q-x9tMWIv9cueOFTFA
"Code! Programming with p5.js" by "The Coding Train"
- /search?q=The+Coding+Train&type=playlist
- /channel/UCvjgXvBlbQiydffZU7m1_aw/playlists
girst [Wed, 5 Jan 2022 21:00:18 +0000 (22:00 +0100)]
port videoid_from_thumbnail() to |G()
girst [Wed, 5 Jan 2022 20:51:50 +0000 (21:51 +0100)]
search results: only parse first itemSectionRenderer
no idea why i switched that around (no comment in
eaa6b44 about it
either), so reverting that. let's us remove flatten() now.
girst [Wed, 5 Jan 2022 20:24:06 +0000 (21:24 +0100)]
don't throw 500 error when logging unknown card type
oops :D
girst [Wed, 5 Jan 2022 20:22:31 +0000 (21:22 +0100)]
port parse_channel_items() to |G()
and remove now obsolete listfind(), first(), findall()
girst [Wed, 5 Jan 2022 19:52:19 +0000 (20:52 +0100)]
don't break when video in /channel/ has no title
apparently, a gridVideoRenderer item can have no 'title' key.
found in the wild on:
- /channel/UChi1hV75qQtKOHatzOWKk5A/ (twice!)
- /channel/UC07djtPESgvXX0B1UFjW9jA/
girst [Tue, 4 Jan 2022 19:58:08 +0000 (20:58 +0100)]
hackily add back publish date to /watch
this is relatively hidden intentionally, since we don't want to
encourage too many requests to the show=meta endpoint.
girst [Mon, 3 Jan 2022 16:12:57 +0000 (17:12 +0100)]
expose additional metadata link in 'more actions' section
also emojifies invidious/youtube links there
girst [Wed, 15 Dec 2021 20:37:29 +0000 (21:37 +0100)]
fix reddit 'more' button
girst [Wed, 15 Dec 2021 19:29:47 +0000 (20:29 +0100)]
implement /watch?show=meta to get metadata removed from ANDROID api
girst [Wed, 15 Dec 2021 19:23:40 +0000 (20:23 +0100)]
prefer microformat.publishDate
applies to g_v_i(metaOnly). videoDetails.publishDate is "0" for premiere
videos.
girst [Wed, 15 Dec 2021 15:52:45 +0000 (16:52 +0100)]
remove rating and thumbs up/down, as that is now removed from the response
girst [Wed, 15 Dec 2021 15:29:25 +0000 (16:29 +0100)]
use correct published time for database when available
now, premiere videos are properly crawled_at=now(); published_at=soon()
girst [Wed, 15 Dec 2021 15:07:14 +0000 (16:07 +0100)]
[DATABASE CHANGE: Migration below] mark premiering videos with 'SOON' badge
their premiering date is now captured correctly, too.
ALTER TABLE videos ADD COLUMN premiere BOOLEAN DEFAULT 0;
girst [Thu, 2 Dec 2021 23:33:39 +0000 (00:33 +0100)]
handle missing averageRating
seems to be absent from responses now sometimes
girst [Sat, 20 Nov 2021 13:33:27 +0000 (14:33 +0100)]
try /c/ before /user/ when canonicalizing user names
especially for <instance_host>/<something>; this returned the wrong
channel, if there is both a /c/ and a /user/ with the same id.
example:
c/BoyBoyProductions => UC_S45UpAYVuc0fYEcHN9BVQ (would be correct)
user/BoyBoyProductions => UC2ct7L3mSnprF5o0GAWLzrg (this was selected)
girst [Sat, 20 Nov 2021 13:27:36 +0000 (14:27 +0100)]
tell the user to log in to view livestreams/geolocked videos, if applicable.
girst [Sat, 20 Nov 2021 13:14:59 +0000 (14:14 +0100)]
allow limiting proxy to authenticated users
g.proxy_on should be accessed through getattr(g, 'proxy_on', None).
This allows for the following states:
True: proxy can be used
False: proxy use is denied; try logging in
None: proxy is disabled globally
girst [Sat, 13 Nov 2021 23:27:59 +0000 (00:27 +0100)]
expose reddit sort orders to users
girst [Sat, 13 Nov 2021 22:35:40 +0000 (23:35 +0100)]
ask for forgiveness when finding video_url
previously, this caused exceptions on utils.py pull, probably due to
scheduled videos, but haven't investigated that.
girst [Sat, 13 Nov 2021 22:27:53 +0000 (23:27 +0100)]
undo reddit continuation/pagination harmonisation
girst [Sat, 13 Nov 2021 22:25:37 +0000 (23:25 +0100)]
macros.pagination -> macros.more
hard-codes label text, removes obsolete direction marker
girst [Sat, 13 Nov 2021 22:15:21 +0000 (23:15 +0100)]
replace querystring_page() by jinja2 code inside the macro
removes handling of obsolete special cases and prev-page-handling. also
remove back button from subscriptions page, since nothing else uses that
feaure any more.
note: {% set _nil = ... %} simulates a {% do ... %} expression, which
isn't available by default in flask's jinja config.
girst [Sat, 13 Nov 2021 21:31:09 +0000 (22:31 +0100)]
allow linking to timestamps with macros.card; use it in reddit/
girst [Sat, 13 Nov 2021 21:29:22 +0000 (22:29 +0100)]
harmonize 'more' button between browse/* and reddit/
now only the subscription feed uses the special features of the
pagination macro. preparation for getting rid of this.
girst [Sat, 13 Nov 2021 21:19:45 +0000 (22:19 +0100)]
only show 'more' button on reddit
mimic recent changes in /browse/
girst [Sat, 13 Nov 2021 21:17:37 +0000 (22:17 +0100)]
port search to new api; use continuation tokens here too
girst [Sat, 13 Nov 2021 20:27:54 +0000 (21:27 +0100)]
allow specifying fetch_ajax() endpoint and make payload configurable
in preperation to move search to new api
girst [Sat, 6 Nov 2021 13:09:48 +0000 (14:09 +0100)]
add NO-URL error description
girst [Sun, 17 Oct 2021 23:43:41 +0000 (01:43 +0200)]
use continuation token instead of manually paginated results
this is how youtube does it, and this also works for channel?sort=oldest
girst [Sat, 16 Oct 2021 16:21:22 +0000 (18:21 +0200)]
improve makefile
girst [Fri, 15 Oct 2021 03:39:49 +0000 (05:39 +0200)]
commit database change to setup.sql
girst [Fri, 15 Oct 2021 03:23:16 +0000 (05:23 +0200)]
makefile: automatically generate hmac secrets for config.ini
girst [Fri, 15 Oct 2021 02:52:50 +0000 (04:52 +0200)]
describe what each module does in advance install section
girst [Fri, 15 Oct 2021 02:42:48 +0000 (04:42 +0200)]
enable proxy by default
girst [Fri, 15 Oct 2021 02:37:31 +0000 (04:37 +0200)]
listen on port 8080 by default to encourage reverse-proxying
girst [Fri, 15 Oct 2021 02:30:07 +0000 (04:30 +0200)]
update installation instructions
girst [Fri, 15 Oct 2021 01:39:41 +0000 (03:39 +0200)]
provide a semi-automatic install mechanism
girst [Fri, 15 Oct 2021 01:09:02 +0000 (03:09 +0200)]
use systemd timers instead of cronjobs
girst [Fri, 15 Oct 2021 01:07:50 +0000 (03:07 +0200)]
remove webhooks gunicorn
this is too much complexity for a default install. running webhooks
externally will still be supported (as that's what s.gir.st will
continue to do).
girst [Fri, 15 Oct 2021 01:05:19 +0000 (03:05 +0200)]
allow running setup.sql/guest.sql multiple times
girst [Fri, 15 Oct 2021 02:13:52 +0000 (04:13 +0200)]
[DATABASE CHANGE: Migration below] implement on-line user creation
ALTER TABLE users ADD COLUMN is_admin BOOLEAN DEFAULT 0;
UPDATE users SET is_admin = 1 WHERE name = "admin"; -- replace with your username
girst [Thu, 14 Oct 2021 21:39:44 +0000 (23:39 +0200)]
remove now unused cipher fetching code
now independent of jwz's youtubedown :^)
girst [Thu, 14 Oct 2021 21:35:25 +0000 (23:35 +0200)]
switch to ANDROID player API
provideds unthrottled and even unscrambled streams. we do have to give
up some metadata: category, availableCountries, infocards, published.
girst [Sat, 18 Sep 2021 19:01:20 +0000 (21:01 +0200)]
redirect thumnail urls
requests to these urls are probably caused by invidious-redirect
extensions.
girst [Sat, 4 Sep 2021 09:50:00 +0000 (11:50 +0200)]
take full youtube.com urls as path, and redirect accordingly
this version only handles redirections to /watch for now.
girst [Mon, 30 Aug 2021 05:40:47 +0000 (07:40 +0200)]
add method to allow downloading though yt-dlp
https://github.com/iv-org/invidious/pull/1730
https://github.com/yt-dlp/yt-dlp/commit/
df0c81513e0bb37986d00c532a5ad8cef31a24ea
girst [Tue, 17 Aug 2021 13:43:29 +0000 (15:43 +0200)]
split stream_map.adaptive into *_video and *_audio
girst [Tue, 17 Aug 2021 12:48:40 +0000 (14:48 +0200)]
fix try_int() to actually return something
urgh.
girst [Fri, 13 Aug 2021 18:14:56 +0000 (20:14 +0200)]
avoid contacting innertube api when video id is obviously wrong
girst [Fri, 13 Aug 2021 18:02:08 +0000 (20:02 +0200)]
avoid int(request.args.get('page'))
this throws a ValueError when ?page=abc. the replacement returns a
default value, or None instead.
girst [Fri, 13 Aug 2021 17:41:01 +0000 (19:41 +0200)]
fix ?show=audio for unplayable videos
they might have metadata, but can still be UNPLAYABLE.
girst [Fri, 13 Aug 2021 17:27:47 +0000 (19:27 +0200)]
handle the case when we can't connect to googlevideo
note: we should also verify the ?mn param's contents
girst [Fri, 13 Aug 2021 17:20:19 +0000 (19:20 +0200)]
fix url extraction of non-embeddable livestreams
test id: 31NX4zpsKuI
girst [Fri, 13 Aug 2021 15:12:13 +0000 (17:12 +0200)]
allow /embed/ urls with final /
girst [Thu, 5 Aug 2021 16:25:15 +0000 (18:25 +0200)]
handle geolocked videos not available in the server's region
example video: cjPWJycHImg
girst [Thu, 5 Aug 2021 15:12:14 +0000 (17:12 +0200)]
make channel page sorting link(-ish) instead of a select box
girst [Fri, 30 Jul 2021 09:34:32 +0000 (11:34 +0200)]
use same embedder-url as yt-dlp
https://github.com/yt-dlp/yt-dlp/commit/
5dcbfdc3cdcfc1f8d70515197d1f6dfcbb28251a