A video encoding with a restart is split into separate files

[Matte27]

No description provided.

[clementgallet]

This is an expected behavior and won't be changed. The encoding is done by the game, so when it exits, the connection to ffmpeg is closed, so is the encode file. And I'm guessing resuming an encode is not possible for the majority of codecs.

[vadosnaprimer]

It's trivial to splice them in post-production.

[Matte27]

Trivial or not, it still takes time to do that. Having it done in the tas tool would make it a lot simpler and faster.

[vadosnaprimer]

How much time does it take exactly?

[Matte27]

I don't know because I haven't done it but I'm sure is a non-trivial amount. Even if I did I don't think it would be less than say 10s, and when I am testing new stuff and looking at the recordings those seconds add up.

[vadosnaprimer]

@clementgallet what's the current naming convention for dump segments?

[clementgallet]

If the name of the file is video.avi, then the first segment is video.avi, the second one is video_1.avi, third is video_2.avi. It splits on the last . character to add the segment number.

[vadosnaprimer]

@Matte27 see how long this takes (install mkvtoolnix, change "avi" to whatever video format you dump libtas videos in)

ls -v *.avi | xargs | { read f; mkvmerge -o output.mkv ${f// / + }; }

[Matte27]

Do you think all TASers will figure out quickly how to do this? Many of whom aren't even familiar with linux. I think not.

[vadosnaprimer]

At first your point against doing this manually was that it's slow. Now you're refusing to even test it and switch to a new argument that holds no water either. Putting this command right into libTAS documentation and readme costs zero effort and makes it instantly obvious for everyone who cares enough to read.

[nlburgin]

Do you think all TASers will figure out quickly how to do this? Many of whom aren't even familiar with linux. I think not.

Encoding being dead-easy was never a hard requirement anyway. A lot of TAS tools don't even have FFMPEG integration at all, but will only dump a massive raw avi that you have to re-encode yourself. At least this takes care of that for you.

And, as has just been pointed out, putting the relevant command right in the documentation would mean they still wouldn't really have to "figure it out" themselves anyway. That command is copy-pasteable and can be re-used as-is as long as you don't have any other avi files in the target folder.

...though on the other hand, with the command being so simple, I'm actually wondering why libtas couldn't just run it automatically at the end of the encoding. 🤔

It wouldn't have to actually avoid splitting, just offer an option to automatically merge the dumps with mkvmerge or ffmpeg -c copy at the end of the process. I don't think that would be terribly hard to implement.

[Matte27]

My point is that it takes some time to splice the segments manually whether you are doing it for the 1st time or 100th time and I don't see why the tas tool wouldn't do it. Yes people would get by without it but having it would be very convenient.

[InfamousKnight]

A script can be made to combine the segments. I'll look into this.

[qixils]

vadosnaprimer already posted a script, unless you want to make a script specifically for ffmpeg..

[nlburgin]

vadosnaprimer already posted a script

He gave a simple example command, not a robust script intended for release.

Ideally an officially released script would be a bit more flexible and robust than just running against "*.avi" (every avi file in the working directory), but be smart enough to specifically pull in only the chunks from a specific encoding job. Also it would be preferable if it could be integrated with the interface in some way.

unless you want to make a script specifically for ffmpeg..

It would probably be preferable if the script could use ffmpeg instead of mkvmerge, since ffmpeg is already a dependency of libTAS but the mkvtoolnix suite is not.

ffmpeg can do the same thing, though the syntax is more complex.

Here's a rough prototype. I haven't tested it with actual output from libTAS, but did test it on a set of videos that was set up to match the naming convention @clementgallet described earlier, and it seems to work.

EDIT: Actually, it only works if there's less than ten. I think '*' globbing has to go, it doesn't guarantee the correct order.

---

#!/bin/bash

# absolute path to the first video (eg. /home/user/idk/video.avi) is passed as command line argument 1
# the file extension (.avi or whatever) is passed as command line argument 2

VID_PATH=$1
VID_EXT=$2
VID_DIR=$(dirname $VID_PATH)
VID_NAME=$(basename -s $VID_EXT $VID_PATH)

getChunks(){
    #print first one outside of the loop, otherwise the * glob will put it out of order.
    echo file $VID_PATH
    
    for i in "$VID_DIR"/"$VID_NAME"_*"$VID_EXT"
    do
      echo file $i
    done
}

ffmpeg -f concat -safe 0 -i <(getChunks) -c copy $VID_DIR/$VID_NAME-merged.mkv

with this set up in concat.sh, you would run something like:

./concat.sh /path/to/video.avi .avi

Attachment :
concat.sh.zip

[nlburgin]

Okay, I think I fixed it now.

This version does them in the right order even if there's more than ten.

It takes much better care of the fact that star-globs don't seem to guarantee the right order. It has to resort to even more obscure bash syntax, but it makes sure everything gets sorted.

Just note this will definitely need actual GNU Bash, lightweight shells like Dash or BusyBox almost certainly won't work.

---

#!/bin/bash

# absolute path to the first video (eg. /home/user/tasproject01/video.avi) is passed as command line argument 1
# the file extension (.avi or whatever) is passed as command line argument 2

VID_PATH=$1
VID_EXT=$2
VID_DIR=$(dirname $VID_PATH)
VID_NAME=$(basename -s $VID_EXT $VID_PATH)

getChunks(){
    #first video is a special case
    echo file $VID_PATH
    
    #sort command can ensure the rest are in order
    temp_var="$VID_DIR"/"$VID_NAME"_
    for i in $(sort -nk 1.$(( ${#temp_var} + 1))  <(ls -1 "$VID_DIR"/"$VID_NAME"_*"$VID_EXT"))
    do
      echo file $i
    done
}

ffmpeg -f concat -safe 0 -i <(getChunks) -c copy $VID_DIR/$VID_NAME-merged.mkv

concat.sh-fixed.zip

[vadosnaprimer]

My line fixes the broken order, I just don't remember anymore which part does what.

[nlburgin]

Well as long as that's the case, yours is definitely better as a copy-paste solution for the end-user, though it would still need a warning to make sure there are no unrelated avi files in the same folder.

However, my intent with the Big Fat Script was more in hopes that the GUI could maybe offer an option of running it automatically at the end of an encode. That seems ideal, if it's feasible.

I don't know how much trouble it is to call bash scripts from C++... I just learned that execl() apparently can directly call scripts that have "!#" set up properly, so it's script-ness is no obstacle. There's probably still a bit of fork/exec/wait boilerplate that would be needed to run it properly, but I'm pretty sure it would still be very little trouble compared to actually doing everything the script does from C++.

[nlburgin]

I just set up a pull request (#219) proposing a solution to this.

The script is a bit modified from what I posted earlier.

[nlburgin]

My line fixes the broken order, I just don't remember anymore which part does what.

I figured out, it's apparently the "ls -v".

Apparently the '-v' option makes it smart about sorting by embedded numbering.

[vadosnaprimer]

xargs was necessary for ordering too.