How to stabilize shaky video on Linux?

I’ve been looking around for Linux video stabilization for about two years. It’s not like the digital stabilization haven’t been present two years ago, but just recently I have been pushed enough that I had to to something about it.

So I need (and needed since two years ago) to stabilize my diving footage. Recording underwater video without some gimbal, because of water movement and my movement, is always quite shaky. It’s event worst when you snorkel, since you usually do a lot of rapid movement. Anyway, I’ve been using to post process video Kdenlive (and had some short episode with Openshoot). Theoretically there is an option to stabilize clip (right click->process) but it haven’t been working on Debian for a long time. Now it does work, but Melt (Mlt) for Debian (I’m talking about Sid/Unstable branch) is compiled without vid.stab, with just old and unsupported videostab and videostab2. Both does not work good enough for me (or I was simply unable to find proper values). Second thing is, that Kdenlive do it in quite annoying way. You have to process clip, so Mlt will create file with all stabilization requirements on per frame basis – but not stabilize clip itself. This will also add a new clip to you project „something.mlt”. You can use this new clip on your time line – like every other clip – but you won’t see stabilization effect until you render your project. And, I don’t know if it’s a bug or feature, you won’t see any other effect/filter put on this new clip, until you render you entire project. So not acceptable for me.

Long story short, I’ve tried Melt + vid.stab – it still had some issues and I choose to add vid.stab to ffmpeg. It’s easy and you can use one single binary (ffmpeg) for entire process along with original ffmpeg package on your Debian.

So you need some basics dev libraries on your Debian, you need to get and compile vid.stab and ffmpeg. This may look like this:

sauron:/tmp# git clone https://github.com/georgmartius/vid.stab
Cloning into 'vid.stab'...
remote: Counting objects: 1403, done.
remote: Total 1403 (delta 0), reused 0 (delta 0), pack-reused 1403
Receiving objects: 100% (1403/1403), 541.49 KiB | 1.04 MiB/s, done.
Resolving deltas: 100% (929/929), done.
sauron:/tmp# cd vid.stab/
sauron:/tmp/vid.stab# cmake .
-- The C compiler identification is GNU 7.2.0
-- The CXX compiler identification is GNU 7.2.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- vidstab: writing pkgconfig file /tmp/vid.stab/vidstab.pc
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/vid.stab
sauron:/tmp/vid.stab# make
Scanning dependencies of target vidstab
[ 7%] Building C object CMakeFiles/vidstab.dir/src/frameinfo.c.o
[ 15%] Building C object CMakeFiles/vidstab.dir/src/transformtype.c.o
[ 23%] Building C object CMakeFiles/vidstab.dir/src/libvidstab.c.o
[ 30%] Building C object CMakeFiles/vidstab.dir/src/transform.c.o
[ 38%] Building C object CMakeFiles/vidstab.dir/src/transformfixedpoint.c.o
[ 46%] Building C object CMakeFiles/vidstab.dir/src/motiondetect.c.o
[ 53%] Building C object CMakeFiles/vidstab.dir/src/motiondetect_opt.c.o
[ 61%] Building C object CMakeFiles/vidstab.dir/src/serialize.c.o
[ 69%] Building C object CMakeFiles/vidstab.dir/src/localmotion2transform.c.o
[ 76%] Building C object CMakeFiles/vidstab.dir/src/boxblur.c.o
[ 84%] Building C object CMakeFiles/vidstab.dir/src/vsvector.c.o
[ 92%] Building C object CMakeFiles/vidstab.dir/src/orc/motiondetectorc.c.o
[100%] Linking C shared library libvidstab.so
[100%] Built target vidstab
sauron:/tmp/vid.stab# make install
[100%] Built target vidstab
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/include/vid.stab/boxblur.h
-- Installing: /usr/local/include/vid.stab/frameinfo.h
-- Installing: /usr/local/include/vid.stab/libvidstab.h
-- Installing: /usr/local/include/vid.stab/localmotion2transform.h
-- Installing: /usr/local/include/vid.stab/motiondetect.h
-- Installing: /usr/local/include/vid.stab/motiondetect_internal.h
-- Installing: /usr/local/include/vid.stab/motiondetect_opt.h
-- Installing: /usr/local/include/vid.stab/serialize.h
-- Installing: /usr/local/include/vid.stab/transform.h
-- Installing: /usr/local/include/vid.stab/transform_internal.h
-- Installing: /usr/local/include/vid.stab/transformfixedpoint.h
-- Installing: /usr/local/include/vid.stab/transformfloat.h
-- Installing: /usr/local/include/vid.stab/transformtype.h
-- Installing: /usr/local/include/vid.stab/transformtype_operations.h
-- Installing: /usr/local/include/vid.stab/vidstabdefines.h
-- Installing: /usr/local/include/vid.stab/vsvector.h
-- Installing: /usr/local/lib/libvidstab.so.1.1
-- Up-to-date: /usr/local/lib/libvidstab.so
-- Installing: /usr/local/lib/pkgconfig/vidstab.pc
sauron:/tmp/vid.stab#

Now we compile ffmpeg

sauron:/tmp# git clone https://github.com/FFmpeg/FFmpeg
Cloning into 'FFmpeg'...
remote: Counting objects: 539951, done.
remote: Total 539951 (delta 0), reused 0 (delta 0), pack-reused 539951
Receiving objects: 100% (539951/539951), 200.29 MiB | 12.06 MiB/s, done.
Resolving deltas: 100% (420867/420867), done.
sauron:/tmp# cd FFmpeg
sauron:/tmp/FFmpeg# ./configure --enable-gpl --enable-libvidstab --enable-libopencv --enable-libx264 --enable-libwebp --enable-libx265

This will, as a result, write you a table of all codecs, filters that are going to be compiled. Check if vidstab is present: vidstabdetect, vidstabtransform

sauron:/tmp/FFmpeg# make

few minutes later...

sauron:/tmp/FFmpeg# ls -lh ffmpeg
-rwxr-xr-x 1 root root 19M Jan 16 11:17 ffmpeg


sauron:/tmp/FFmpeg# ./ffmpeg -filters 2>/dev/null | grep vidst
 ... vidstabdetect V->V Extract relative transformations, pass 1 of 2 for stabilization (see vidstabtransform for pass 2).
 ... vidstabtransform V->V Transform the frames, pass 2 of 2 for stabilization (see vidstabdetect for pass 1).

sauron:/tmp/FFmpeg# ./ffmpeg -codecs 2>/dev/null | grep h264
 DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_v4l2m2m )

sauron:/tmp/cos/FFmpeg# mv ffmpeg /usr/local/bin/
sauron:/tmp/cos/FFmpeg#

Two almost last lines with -filters and -codecs, check if there is indeed vid.stab build in and if we have a output h264 codec (this is important „encoders: libx264”)
Now you have ffmpeg binary in /usr/local/bin and (if you already had installed ffmpeg package) another one in /usr/bin.

How to stabilize video?

Stabilization process with vid.stab is made in two passes. First pass, filter will generate file with all informations about „how to stabilize” video, according to provided settings. Second pass, will do actual stabilization and render file.

To make it easier, I’ve made a short script – just put it in /usr/local/bin/ and provide file name as an argument.

sauron@sauron:~$ cat /usr/local/bin/stabilize_video.sh

#!/bin/bash

if [[ "$1" == "" || "$1" == "help" ]]
then
 echo "Usage: `basename $0` filename" >&2
 # Error message to stderr.
 exit $E_NOARGS
 # Returns 85 as exit status of script (error code).
fi

if [ ! -f "$1" ]; then
 echo "Error: Selected file ($1) is missing"
 exit
fi

if [[ "$1" != *.* ]]; then
 echo "Error: File has no extension (example.mkv)"
 exit
fi

FILE=$1
FILE_BASE=`echo -n "$1" | cut -d "." -f 1 -`
FILE_TRF=$FILE_BASE".trf"
FILE_OUT=$FILE_BASE"-stab.mkv"

echo "Processing file:"
echo -e "\t input file: "$FILE
echo -e "\t pass 1 file: "$FILE_TRF
echo -e "\t output file: "$FILE_OUT

sleep 2

ffmpeg -i $FILE -vf vidstabdetect=shakiness=10:accuracy=15:result="$FILE_TRF" -f null -
ffmpeg -i $FILE -vf vidstabtransform=input="$FILE_TRF",unsharp=5:5:0.8:3:3:0.4 -c:v libx264 -crf 16 -c:a copy -preset fast $FILE_OUT
And now some explanation – important are two last lines with proper invocation of ffmpeg.
First we need to learn how the video is build – and generate the knowledge base. We can influence on process with few parameters how „shaky” video is and how much work algorithm should involve – there are some other options, that you can check in README file. Shakiness=10 means video is quite a lot unstable.
Next we need to generate stable video. We will use vid.stab, add some sharpness to image and output it as h264 video codec with quite generous CRF=16 (video is going to be quite large – but in my case, this are only temporary files) and fast preset (this only influence on file size, not file quality). We do not touch audio track.

How to compare two video tracks?

Now the fun part, how to see how well it was done? You could (they say so 😉 ) vlc and open two video streams, but this does not work for me (vlc=3.0.0). So we can, once again, use ffmpeg goodies. This time ffplay.
And another short script 🙂
sauron@sauron:~$ cat /usr/local/bin/compare_video.sh
#!/bin/bash

if [[ "$1" == "" || "$1" == "help" || "$2" == "" ]]
then
 echo "Usage: `basename $0` video1 video2 [hh:mm:ss]" >&2
 # Error message to stderr.
 exit $E_NOARGS
 # Returns 85 as exit status of script (error code).
fi

if [[ ! -f "$1" || ! -f "$2" ]]; then
 echo "Error: Selected file(s) ($1) are missing"
 exit
fi

if [[ "$1" != *.* ]]; then
 echo "Error: File has no extension (example.mkv)"
 exit
fi

FILE1=$1
FILE2=$2
STIME="00:00:00"

if [[ "$3" == ??:??:?? ]]; then
 STIME=$3
fi

echo "Processing files:"
echo -e "\t input file1: "$FILE1
echo -e "\t input file2: "$FILE2
echo -e "\t starting at: "$STIME

sleep 2

ffplay -i $FILE1 -ss $STIME -vf "[in] scale=iw/2:ih/2, pad=2*iw:ih [left];movie=$FILE2, scale=iw/2:ih/2 [right];[left][right] overlay=main_w/2:0 [out]"
Just run the script with two files original and stabilized one as argument. There can be some problems with video seeking (it may not work :)). If you wan to, you can switch ffplay to ffmpeg to generate output file (like in youtube example).
If you have any insight in other way to stabilize footage (on Linux 🙂 ), or other working settings for vid.stab – please share in comments!

Dodaj komentarz

Bądź pierwszy!

avatar
  Subscribe  
Powiadom o