2019-12-14

rsync

Synonyms:

SSH Transfer Files Using rsync

rsync can be used to synchronize files between two Ubuntu servers. It is more complicated than scp.

Prerequisite

SSH connect using the same username without password as SSH Connect from Ubuntu said.
Login to {{ username.value }} account of server A.

Synchronize Directory

Push Directory from Server A to Server B

The following files/directories in Server A will be excluded:

{{ item }}

Files/directories exist in Server B but aren't present in Server A will be deleted, with exception of the following patterns: {class="red-text"}

{{ item }}

sudo rsync -avz -e "sudo -u {{ username.value }} ssh" --rsync-path="sudo rsync" {{ option }} {{ a_path.value }} {{ b_hostname.value }}:{{ b_path.value }}

Pull Directory from Server B to Server A

The following files/directories in Server B will be excluded.

{{ item }}

Files/directories exist in Server A but aren't present in Server B will be deleted, with exception of the following patterns: {class="red-text"}

{{ item }}

sudo rsync -avz -e "sudo -u {{ username.value }} ssh" --rsync-path="sudo rsync" {{ option }} {{ b_hostname.value }}:{{ b_path.value }} {{ a_path.value }}

parallel by directories (Pull Directory from Server B to Server A)

sudo rsync --dry-run --filter "+ /*/" --filter "- *" -avz -e "sudo -u {{ username.value }} ssh" --rsync-path="sudo rsync" {{ option }} {{ b_hostname.value }}:{{ b_path.value }} {{ a_path.value }} > /tmp/transfer.log
sed -i '1d' /tmp/transfer.log
sed -i '$d' /tmp/transfer.log
sed -i '$d' /tmp/transfer.log
sed -i '$d' /tmp/transfer.log

# 在删除一部分？(需要修改)
#sed -i '/\/$/d' /tmp/transfer.log

# dry-run测试
tail /tmp/transfer.log
cat /tmp/transfer.log | parallel --will-cite --dry-run -j {{ job.value }} sudo rsync -avz -e \"sudo -u {{ username.value }} ssh\" --rsync-path=\"sudo rsync\" {{ option }} {{ b_hostname.value }}:{{ b_path.value }}{} {{ a_path.value }}{} > /tmp/result.log
tail /tmp/result.log

cat /tmp/transfer.log | parallel --will-cite -j {{ job.value }} sudo rsync -avz -e \"sudo -u {{ username.value }} ssh\" --rsync-path=\"sudo rsync\" {{ option }} {{ b_hostname.value }}:{{ b_path.value }}{} {{ a_path.value }}{} > /tmp/result.log
tail /tmp/result.log

Reference

rsync Filter Rules

Search FILTER RULES in man page of rsync.