Feeling powerful with FFmpeg and Imagemagick
Introduction
Recently, I took the task of writing the user documentation for the FOSS United Dashboard. It’s an event management dashboard made available to all the city chapter volunteers who can CRUD manage FOSS Meetups, Events, and Hackathons. Ngl, it was good and it is indeed very hard to write good documentation, I did terrible work. It slowly felt unexcited, so I thought of making some asset in it exciting
A lot of my friends on Telegram talked about FFMPeg when I was around 15 or 16 and I had no clue what it was, from the last 1 year I’ve been trying to bring FFMPeg and Imagemagick in hand. In this blog, I want to share some snippets or scripts on using FFmpeg and ImageMagick daily.
Imagemagick
The grudge was to take beautiful screenshots from a long time. MacOS has good inbuilt proprietary tools to beautify a screenshot. In the Ricers community, there have been many people who did that with the help of ImageMagick and I had done this before as well. I’ve been very lazy for a long time and I was manually capturing screenshots whenever needed. 2 weeks ago I decided I’d nail it down this time and fixed my i3 config to automate taking screenshots by applying the ImageMagick effect in it.
demo of how my screenshots look
I use i3-gaps as my window manager, so I’ve configured keys to take screenshots with maim, and then in the pipeline magick does its magic of adding a dark background and some drop shadow to the image. The command is as follows
maim -s --hidecursor | magick - \( +clone -background black -shadow 50x50 \) +swap -background "#141b1e" -layers merge +repage ~/Pictures/Capture-$(date +%h-%d-%H-%M-%S)
for selecting the area you want to take screenshots off
maim -d 3 --hidecursor | magick - \( +clone -background black -shadow 50x50 \) +swap -background "#141b1e" -layers merge +repage ~/Pictures/Capture-$(date +%h-%d-%H-%M-%S)
for taking a full-screen screenshot.
And to make them work automatically I have bound them with keys in the i3 window manager to provide ease. You can also pipe clip to copy the image to your clipboard and use tesseract if you you need text recognition and OCR.
ImageMagick can do more exciting stuff, I use it almost whenever I want to grayscale, rotate, or convert any of my screenshots or photographs
FFmpeg
So, for those of who you don’t know, let me tell you - FFmpeg is the real chad, FFmpeg is literally a video editor on the command line. Again, I use FFmpeg almost everywhere possible be it trying to convert FLAC audio into various sizes of opus or vice versa, trimming videos, or recording my screen (it is a pain, but try).
Getting back to the original story, I also had to record gifs or tutorials to add ease for the volunteers to understand the workings of our dashboard manager. So, I thought why not also make a gif of a similar style to the ImageMagick screenshot I take? It got pretty exciting and I could feel it. I had previously seen how beautiful and elegant MacOS screen recordings look.
The end goal
The same command can’t be applied on a GIF or mp4 format video of course, because it’s common sense that ImageMagick only does magick on image formats. I was parallelly talking in a telegram group about all this and Mani sir showed up and gave me the idea that I could extract each frame from a screenshot, then searching started and I found some commands which were doing what I wanted. I’ll break it down below
Firstly, extracting all the frames from the mp4 file (GIF won’t give you HQ, so don’t get fooled) but no one is stopping you from extracting from a GIF
ffmpeg -i <file>.mp4 -vf fps=12 out%d.png
extracting in 12+ FPS is recommended if you have a great CPU and a heavy resource computer, because, in the next steps, it will take you an eternity to do the stuff coming ahead. So, if one mp4 has 30 frames at 12 FPS then 12*30 = 350 photos.
The MP4 I had was around 30-60sec (I don’t remember correctly) but after extracting I got 490 frames :D and it got painful later. IIRC I had set fps=8
I made a very simple script called meckos for the purpose of converting a normal image into the Magick version with a background and drop shadow effect.
for i in *.png ; do meckos $i converted_$i ; done
With this for loop, I get all the frames converted into a magick version. Converting 480 frames took a lot of time and my laptop started heating a bit. This was because I did not do any kind of optimization you can do more optimization in the above command by running the processes parallelly.
It’s time you bring every back together in a tight brotherhood bond lol.
ffmpeg -i converted_out%d.png output.mp4
With this command, all of the frames will turn back into an MP4 video, but it will be very fast in the first go and won’t look very pleasing. For that, we have to increase the speed of the video. The commands here were given to me by Arun as he found this, so it’s great thanks to him. He was there with me in the whole process guiding through and tinkering with me.
ffmpeg -i <file>.mp4 -map 0:v -c:v copy -bsf:v hevc_mp4toannexb raw.h265
So, whatever is available on the top level of the internet is usually heavily compressed, and a lot of details in it are lost be it (information or) media. raw.h264 or raw.h265 here is the lossless version of videos, cameras usually record footage in the RAW () Codec which provides seamless and lossless video at its highest capability in quality. HEVC (High-Efficiency Video Coding) is the international standard of Video Compression where the quality is very high and exactly the same as 4k.
So, after the above command, we get the raw.h265 file which is the highest quality video I have of my fast af video. This step is essential to convert your video into a faster one, because if you slow down your video on the fastened video then it is likely to lose quality.
The final step, bring it to the speed of video so that it looks normal for humans lol.
ffmpeg -fflags +genpts -r 30 -i raw.h264 -c:v copy <output-file>.mp4
-r is used to set the framerate again here, 30 is what I thought should be okay to display, but it was actually -5% of the decent level of video. One of my friends roasted the GIF in one sight.
Conclusion
No conclusion, this was just the first part, I did a hell lot of more things after that which I am not including here as the blog is getting bigger, but were pretty fun as well but it wasn’t working like I wanted to. I was automating this whole process as soon as the recording with ffmpeg was done. So, it was getting very tedious in my system as well as on the event dashboard, because I had to create a new event every time I messed up while shooting the video.
But, this was a very fun experience and I learned a lot about the potential of FFmpeg and Image Magick. Great pieces of software,I fell in love with these now. even though I did not use the MP4 files in the Documentation on the website lol, I’d say all the struggle was worth it, and most importantly I enjoyed the process.
Listing out some resources I read while writing the blog and while making whatever it was
Great thanks to Arun Mani J. Do check his blog on ImageMagick