So: you've got two or more clients and/or servers. They contain files that you want to have automatically synced when possible, because that would save a lot of time. Well, I got the solution for you: with a little bit of thinking in an innovative way I have found the solution that might bring you onto the right path as well. Rsync is a great solution, but having to run rsync manually would take a lot of unnecessary time away, right? And that is where inotify is for: real-time monitoring of your filesystem so that your files can be synced between multiple machines with the power of rsync!
Setting up a test scenario
I needed to get myself a nice development environment at first so I started off 3 virtual servers which all run Ubuntu 16.04, my personal favourite. All these 3 machines needed to be setup with the following software packages:
- OpenSSH server
- Rsync
- Inotify
Also noteworthy is that these machines are absolutely not connected through a private network. All rsync traffic is supposed to be worked out over SFTP.
For who is this for?
I could see some potential for workflow improvement on these situations:
- A development environment, where constant file transfers are taking up a lot and/or too much time
- Load balanced file storage clusters/servers
- Backup/failover servers with the need for constant replication
Working on it
First things first: we need to get all the dependencies installed on the 3 servers with this one-line command:
apt update && apt -y install openssh-server rsync inotify-tools
After that, let's create a specific folder that we want to sync. Let's call it SyncFiles:
mkdir /opt/syncfiles
And for secure file transfer, we want a public-private key link for the transfer link that rsync uses, this is how to configure it:
ssh-keygen -t rsa -f ~/rsync-key -N ''
# Paste the output in your destination servers' ~/.ssh/authorized_keys file:
cat ~/rsync-key.pub
# Removing public key for security purposes..
rm ~/rsync-key.pub
# Remember to execute this script on all servers separately! Then, copy the output of the script in all of your servers' authorized_keys files.
Now that you have got all the pre-configuration work done, it's about time to write a script that goes through an infinite loop with inotifywait in it:
#!/bin/bash
# Supposed to run on rsync-host01, change rsync-host02 to rsync-host01 to make a script that is meant to run on rsync-host02.
while true; do
inotifywait -r -e modify,attrib,close_write,move,create,delete /opt/syncfiles
rsync -avz -e "ssh -i /root/rsync-key -o StrictHostKeyChecking=no“ /opt/syncfiles/ root@rsync-host02:/opt/syncfiles/
done
I saved this script in the /opt directory as file-sync.sh.
To finish things off, lets create the systemd service file that can stop, start, and reset the script on demand or on specific events like a system bootup.
Create a file called sync.service in the directory /etc/systemd/system/ and put the following contents in it:
[Unit]
Description = SyncService
After = network.target
[Service]
PIDFile = /run/syncservice/syncservice.pid
User = root
Group = root
WorkingDirectory = /opt
ExecStartPre = /bin/mkdir /run/syncservice
ExecStartPre = /bin/chown -R root:root /run/syncservice
ExecStart = /bin/bash /opt/file-sync.sh
ExecReload = /bin/kill -s HUP $MAINPID
ExecStop = /bin/kill -s TERM $MAINPID
ExecStopPost = /bin/rm -rf /run/syncservice
PrivateTmp = true
[Install]
WantedBy = multi-user.target
Chmod this service file and reload the systemd daemon:
chmod 755 /etc/systemd/system/sync.service
systemctl daemon-reload
You are all set! You can now use these commands to manage your self-made directory sync daemon:
# Start your service
systemctl start sync.service
# Obtain your services' status
systemctl status sync.service
# Stop your service
systemctl stop sync.service
# Restart your service
systemctl restart sync.service