shell

Automatically Backing Up Docker Data Volumes

by

After much searching, I was unable to find a script that would automatically backup my Docker data volumes. Everything I found required me to manually add container names as I deployed them and I was not interested in doing that. As a result, I wrote my own script that accomplishes this goal.

This script will create an array that contains the names of all the folders under a specific path. Next, the script will see if there is a folder name that matches the name of a container in docker. If a match does not exist, the script will skip performing a backup. If a match does exist, the script will stop that container in docker, check to see if a backup folder exists for that container, and create the folder if it does not exist. The script will then archive that data volume to the backups directory, check to see how many versions exist, and keep only the number specified. Once everything is done, the script uses docker compose to start all containers in the correct order.

Variables

  • docker – path to docker data volumes
  • backups – path to where you keep your backups
  • dcrun – alias to be able to run docker compose commands with less text
  • array – creates an array of all applications listed under specified directory
  • versions – how many versions of backups you want to keep

I am sure there are better ways to do all this, but I chose this strategy since I do everything in docker compose and name the data volumes the same thing as the docker container. I have this script run as a nightly cron job and have duplicati store a copy of these backups a cloud storage provider. Please make sure you save your backups somewhere outside of the server! If you don’t, you risk losing everything if the storage you have the backups saved on dies.

#!/bin/bash

#variables
docker="/opt"
backups="/mnt/storage/backups"
dcrun='docker-compose -f /opt/tools/docker_compose.yml '
array=( ${docker}/* ) #Array of all applications listed underneath /opt
versions="3"

#Loop through all docker apps which have app files in /opt
for path in "${array[@]}"
do
	if [ "$(docker ps -q -f name=${path##*/})" ]; then #checks if app is setup in docker	
		echo "start ${path##*/} backup"
		${dcrun} stop ${path##*/} #stop container using docker compose
		
		#docker stop ${path##*/} #uncomment this line and comment out line above if not using docker compose
		mkdir -p ${backups}/${path##*/} #makes new backup directory if new container is added
		tar -czf ${backups}/${path##*/}/${path##*/}_$(date +"%Y.%m.%d").tar.gz ${path} #take backup of app folder into archive
				ls -rt -d -1 ${backups}/${path##*/}/*| head -n -${versions}| xargs -d '\n' rm -f -- #keeps latest 3 archives in directory
		echo "end ${path##*/} backup"
		echo " "
		echo " "
	fi
done
	
#start docker containers
${dcrun} up -d

echo "backup complete"

Trimming the front and end of a video using Linux

by

I wanted to trim a bunch of videos that I had and wanted an easy and quick way to accomplish this task. I ended up piecing together a small script in order to cut the videos. There are two values you need to change in order to set the time in seconds you want to cut at the start of the script. The following package are needed to run the script

  • ffmpeg
  • bc

Also, be sure to create a sub-folder named “output” under whichever directory you choose to run the script in.

#!/bin/bash

for f in *.mp4; do

	#durations to cut video in seconds
	front=8
	back=15

	#trim 8 second off front of video (ss -8)
	ffmpeg -i "$f" -vcodec copy -acodec copy -ss "$front" ./output/front_"$f"
	
	#calculate total video size
	input_duration=$(ffprobe -v error -select_streams v:0 -show_entries stream=duration -of default=noprint_wrappers=1:nokey=1 "$f")
	#calculate what needs to be cut off end
	output_duration=$(bc <<< "$input_duration"-"$back")
	
	
	#trim the end
	ffmpeg -i ./output/front_"$f" -map 0 -c copy -t "$output_duration" ./output/end_"$f"
	
	#clean up temp file from front trim
	rm ./output/front_"$f"
	
	#remove metadata
	ffmpeg -i ./output/end_"$f" -map_metadata -1 -c:v copy -c:a copy ./output/"$f"
	
	#clean up file from end trim
	rm ./output/end_"$f"

done