UNLOCKING MY DOCO DREAMS FROM DVD

This post isn't just another one of my documentary recommendations, 
but also a tech post. The reason is that it's a documentary that I 
found particularly difficult to get off the internet and into a 
format that's usable to me.

To begin with, the tech part [actually this ended up so long that 
I'll do the documentary review as a separate "part 2"]. This was 
largely a battle with FFmpeg, hampered further by my limited 
capacity for large downloads over my home internet connection which 
required me to do everything on a VPS.

California Dreamin', in documentary form, was a German production 
for their TV station ZDF. Extensive searching revealed this copy 
with German narration on YouTube, possibly with the option of some 
dodgy auto-translated subtitles, but I've given up on that feature:

https://www.youtube.com/watch?v=-s7VXGIgHNU
https://www.youtube.com/watch?v=ZzWLrFharf0
https://www.youtube.com/watch?v=Upa48HW1Gks

Alternatively it's on the Internet Archive, in a straight ZIP 
archive of the files from a DVD release:

https://archive.org/details/2001-california-dreamin-dvd

Even though it was uploaded by IA's best known figure, Jason Scott 
back in 2012, it's only had 319 views. Probably because a ZIP of 
contents from two DVDs is pretty inconvenient to use, and the 
description is entirely in German. Browsing through the ZIP 
contents on the IA website however, I found that it included a scan 
of the DVD cover, in German as well but apparantly claiming to 
include an English ("Englisch") language option.

At 15GB that ZIP would have almost blown my 6 month internet data 
quota on its own, so I relaunched one of my free Oracle Cloud VPSs 
which come with excessive amounts of storage space. Unfortunately 
its installed Ubuntu version had gone EOL while it was 
automatically stopped by Oracle due to disuse, but for this it 
matters little. On there I downloaded the ZIP and decompressed it 
to find apparantly two big VOBs per episode of the five-part 
documentary spread over two DVDs. 'ffprobe' showed that they indeed 
had multiple audio tracks, matching the list of options on the DVD 
cover: German, English, Italian, and French. Easy - 'cat' together 
pairs of VOBs, then select the English audio stream to put into a 
more compressed set of per-episode English-language video files 
small enough for me to download.

First problem is that FFmpeg doesn't show what language the audio 
streams are for, so I had to extract snippets of them as MP3s using 
the '-map' stream-selection option, download, them, then listen to 
the narration after the title music (California Dreamin', of 
course). I found the right one in the first VOB, and of course it 
would be the same stream number for the others right?... Better 
check... nope! In the end it turned out to be something like stream 
5 for the first episode but stream 3 for the others.

So I converted the first episode using my normal options plus 
mapping audio to the English-language stream, downloaded it, and... 
The audio was way out of sync. It was actually a few seconds ahead 
of the video, made obvious by the convenient old-fashioned A/V sync 
numbers/bleeps at the beginning. Worse, I now discovered that 
although it looked like there were two VOBs per episode, they 
weren't actually aligned with episode start/end so I had the start 
of episode two in my episode one file.

The first problem turned out to be due to my usual '-r 24' option 
to set the framerate down to 24fps and avoid wasting data on frames 
I won't notice. The original was 25fps and somehow the frame 
dropping doesn't work right with VOB files, even though it hasn't 
caused me trouble with any other formats. I removed that option and 
the audio sync was correct at 25fps.

I couldn't find an easy answer to the other problem, unable to run 
graphical DVD ripper programs because "SSH -C -X" over my home 
internet is way too laggy and can chew through data itself. But 
eventually I discovered that there's a fork of FFmpeg with extra 
DVD/BR logic added, triggered via a special "dvd://" file type. 
It's called vgtmpeg, and not packaged for Ubuntu so I decided 
"security be dammed" and installed the static binary from its 
webpage (that's what a throwaway VPS is good for anyway).

http://godromo.com/gmt/vgtmpeg

So much easier! Now it can list all the audio streams along with 
their descriptions showing which language they're for. Plus I can 
select titles and they're fetched in the correct duration 
regardless of how the VOBs are laid out. Plus the audio streams 
don't move around between episodes anymore! So I can do the 
conversion like this (with my long sequence of specific audio/video 
output format options setting bitrate etc. removed):

vgtmpeg -i dvd:///home/ubuntu/downloads/California\ Dreamin\'\,\ 2001/Disc\ 1\ -\ Wilder\ Westen\,\ Goldrausch\,\ Wellenreiter/?title=3 -map 0:0 -map 0:2 -acodec mp3 -vcodec mpeg4 california_dreamin_ep1.avi

Or to do all the episodes on the first disc:

for i in 1 2 3; do vgtmpeg -i dvd:///home/ubuntu/downloads/California\ Dreamin\'\,\ 2001/Disc\ 1\ -\ Wilder\ Westen\,\ Goldrausch\,\ Wellenreiter/?title=$[ $i + 2 ] -acodec mp3 -vcodec mpeg4 -map 0:0 -map 0:2 $i.avi; done

Then I got to download all five episodes while using a little under 
1GB of data unstead of 15GB, ready to play on the old hacked video 
game console that I use as a video file player on my TV. It's easy 
once you know how, but oh how much time I could have saved if the 
main FFmpeg project had included this functionality! The fork, 
using a GPL2-licenced library, has been around for many years so 
it's not like the developers haven't had the opportunity. Do I want 
to know why the FFmpeg developers have't integrated this code? 
Probably not.

Finally, it crossed my mind to upload these more convenient video 
files, or higher bitrate ones that others would find more 
acceptable, to the Internet Archive, so other doco lovers needn't 
also go to such trouble (and the IA website's own video player 
would work). Unfortunately though, that's pretty awkward to do from 
the VPS since IA's uploader doesn't work with simple terminal-based 
browsers like Links (indeed not much of their website does 
anymore). There's a special command-line uploader, but it looks 
like it would take some work to figure out using it properly to 
handle the description, uploading multiple files, etc.. Plus now 
I've talked about the doco here I'll have to make a new 'The Free 
Thinker' IA account to do it without harming my anonimity.

If a fellow documentary fan with lots of internet data to burn 
wants to upload them to the Internet Archive for me, send me an 
email at freet@aussies.space and I can provide a download link to 
get them from the VPS. Or just make them yourself, since I've shown 
you how.

 - The Free Thinker