Did some further investigation.
I've build fbpdf (https://github.com/aligrudi/fbpdf, with all the -lmupdf-* removed from the Makefile to make it build) and this works very well.
With pipe-viewer I can watch YT video's (which uses mpv under the hood).
Me. Along with vi depending on my mood.