Coffee Space

Problem

For a long time now, I have avoided the use of Youtube. Many years ago Google acquired Youtube, which generally meant stronger servers and a better experience for users. Over the years, Google have eroded the Youtube platform in favour of big media and advertisers - and rather than be a platform for small creators it has essentially become this neutered ad-friendly version of itself. The main issues I have with the current version of Youtube are:

1. Bloated website - The website is very heavy in JS. At the current time of writing, I have 800+ tabs open and I cannot afford to have needy JS running in the background. Unfortunately, Youtube will completely refuse to operate without it.
2. Very distracting - The recommendations service, whilst being great for them and their revenue, is awful for me and my workflow. I want a distraction free experience and Youtube cannot offer this.
3. RSS OPML export removal - This one really bugged me - I was no longer able to export my subscriptions to RSS. Worse still, Thunderbird corrupted what existing RSS feeds I had. Now the only way to gather the RSS feed for a Youtube channel (for now) is to manually select each channel.
4. Google account login - The last straw was then Youtube completely refusing to allow me to login and comment on videos, if I had any kind of tracking plug-in enabled at all. It had just become an unusable mess.

It was at this time I decided to add the Firefox plug-in Privacy Redirect by Simon Brazell. It performs redirects for services such as Twitter, Youtube, Instagram, Reddit, etc. This allows you to avoid using their main UI and instead use a third-party front-end.

For Youtube, Privacy Redirect relies on third-party hosted instances of Invidious, a pretty robust front-end replacement for Youtube. This for the most part ‘works’, but with some major caveats:

• Timeout - Tabs left open for a while had a terrible habit of timing out - sometimes even whilst mid-watching a video. The result was losing my position in a very long video!
• Bad JS - This one was a major problem for me. When a video failed in the background, it would keep performing DNS requests in the background. For the most part, you may ask “how bad could it possibly be?”. 10MB/s in DNS requests alone. As you may imagine, CloudFlare rightfully temporarily restricted my DNS lookups as a result (having a negative effect on other tabs too), which seemed to only serve to create an infinite loop of requests. Worst still, this was using insane amounts of CPU (a whole core) and RAM (several GBs).

If I am completely honest though, this never really solved my original problems. Rather than relying on Youtube, I ended up relying on services like kavin.rocks, specifically it’s Invidious instance. I don’t know who this person is, what they are doing with my data and why I exactly need to enable their JS in my browser.

Surely we can do better?

Alternatives

My first reaction was “I could code this better”, but I thought also to myself “hmm, you have a bad habit of underestimating exactly how long these projects actually take”. With that in mind, I re-considered what already exists out there.

An alternative to Invidious is NewPipe, as well as something like Piped. Whilst these are very compelling alternatives, these are also massive projects.

Some issues I find with these Youtube alternatives:

• JS - Heavy use of JS that doesn’t scale to many tabs well. I believe a well implemented solution could do away with this entirely!
• Hosting - Requires tonnes of resources! (See languages below.) The servers hosting these services must be insanely good, not something I can afford to host myself.
• Soy languages - What is crystal or vue? Who is seriously using these? How on earth is a JS based language going to be future proof?

Requirements

Here we will discuss two types of requirements, one we will class as ideological requirements and the other we will classify as functional requirements. The difference being that the ideological requirements are general ideas that I want to encapsulate into the project and functional requirements are more formal in terms of basic functionality.

We have the following ideological requirements:

• Privacy - People generally don’t want to be tracked across random platforms. This means not allowing any kind of JS tracking, or even any requirement that JS is run at all!
• Lightweight - An alternative needs to be much lighter than Youtube and Invidious (otherwise why try to replace them?). Ideally it should have little to no impact on CPU/RAM resources until a video is actually playing.
• Distraction-less - It should have a focus on simplicity - this is both easier in terms of design, serving resources and not be a bottomless pit of awful viewing habits.

And the following functional requirements:

• Minimum quality - It needs to be able to watch 480 MP4 videos. I believe that is the minimum that still looks reasonable on a HD display, yet has reasonable resource requirements to download and serve.
• Support old devices - Be able to play on old simple devices (such as old Android). A lot of these older devices only have acceleration for H264 video codecs - anything newer literally cannot be played.
• Remote service - Be able to host the service on a remote server. When the software is stable I want other people to be able to use it too.

Solution

So, all that said, I have written ytoff. The core project is a small script written in Python (main.py) and is supported by a pretty flexible configuration script (default.json). As at the time of writing, cloc reports the line count as:

0001 -----------------------------------------------------------
0002 Language             files      blank    comment       code
0003 -----------------------------------------------------------
0004 Python                   1          7         39        163
0005 JSON                     1          0          0        128
0006 -----------------------------------------------------------
0007 SUM:                     2          7         39        291
0008 -----------------------------------------------------------

It was originally based on youtube-dl, but that was slow and unreliable. I ended up switching to the newer yt-dlp and this offered a massive speed-up ¹.

To get this project off the ground so quickly, I changed away from my usual thinking. Normally I would ask myself: “How can I make this project as future proof as possible?”. This line of thinking leads to an overly complicated project that fails to deliver even on the minimum requirements, eventually getting abandoned as a result. Instead I tackled this project in a different way: “What is the minimum I can do to get this working?”. This approach actually ended up being more fruitful, as I now have something I can work on iteratively. I can always rewrite it at a later date to be more secure or faster, but for now, it will serve the purpose of relaxing my system resources!

Currently I have the server running locally at http://127.0.0.1:3333/, and have Privacy Redirect setup to point all Youtube video links to this URL. This mostly works! There are still a few issues to be resolved as of yet:

1. Metadata - It would be nice to have metadata returned more soon, especially as the queue builds up!
2. Dequeue - If I close a tab before the video was downloaded (I saw the metadata and decided I no longer want to watch it), then it should dequeue the download. (Perhaps this could be based on a “last requested” mechanism.)
3. No formats - Currently if a format cannot be found it can sit in the queue indefinitely. This is far from ideal. Some work can go towards finding other possible formats, but also there should be some mechanism to recognize that we have failed to find a video.
4. Timeout - Somehow I should figure out a way to stop people attacking the server by consuming tonnes of resources (i.e. downloading video ultra-slowly on purpose). The other option could be to allow multi-part downloads and chunk off the responses, so that multiple requests may be made over different connections.

Next Steps

Overall, for a quick hack I am quite happy with this implementation so far. It’s not even that resource intensive!

As previously mentioned, I will maintain my “minimalism” approach - I won’t fix it until there is an issue. There are a few functional items I would like to fix ASAP though for usability, namely shortened URLs and embeds. This should in theory be the only thing stopping me from testing it on a proper server. Watch this space!