1. MagicESP
  2. Ubuntu Server 18.04
  3. SSH Transfer Files Using rsync

SSH Transfer Files Using rsync

Synonyms:
  1. rsync

rsync can be used to synchronize files between two Ubuntu servers. It is more complicated than scp.

Prerequisite

  • SSH connect using the same username without password as SSH Connect from Ubuntu said.
  • Login to {{ username.value }} account of server A.

Synchronize Directory

Push Directory from Server A to Server B

The following files/directories in Server A will be excluded:

  • {{ item }}

Files/directories exist in Server B but aren't present in Server A will be deleted, with exception of the following patterns: {class="red-text"}

  • {{ item }}
sudo rsync -avz -e "sudo -u {{ username.value }} ssh" --rsync-path="sudo rsync" {{ option }} {{ a_path.value }} {{ b_hostname.value }}:{{ b_path.value }}

Pull Directory from Server B to Server A

The following files/directories in Server B will be excluded.

  • {{ item }}

Files/directories exist in Server A but aren't present in Server B will be deleted, with exception of the following patterns: {class="red-text"}

  • {{ item }}
sudo rsync -avz -e "sudo -u {{ username.value }} ssh" --rsync-path="sudo rsync" {{ option }} {{ b_hostname.value }}:{{ b_path.value }} {{ a_path.value }}

parallel by directories (Pull Directory from Server B to Server A)

sudo rsync --dry-run --filter "+ /*/" --filter "- *" -avz -e "sudo -u {{ username.value }} ssh" --rsync-path="sudo rsync" {{ option }} {{ b_hostname.value }}:{{ b_path.value }} {{ a_path.value }} > /tmp/transfer.log
sed -i '1d' /tmp/transfer.log
sed -i '$d' /tmp/transfer.log
sed -i '$d' /tmp/transfer.log
sed -i '$d' /tmp/transfer.log

# 在删除一部分?(需要修改)
#sed -i '/\/$/d' /tmp/transfer.log

# dry-run测试
tail /tmp/transfer.log
cat /tmp/transfer.log | parallel --will-cite --dry-run -j {{ job.value }} sudo rsync -avz -e \"sudo -u {{ username.value }} ssh\" --rsync-path=\"sudo rsync\" {{ option }} {{ b_hostname.value }}:{{ b_path.value }}{} {{ a_path.value }}{} > /tmp/result.log
tail /tmp/result.log

cat /tmp/transfer.log | parallel --will-cite -j {{ job.value }} sudo rsync -avz -e \"sudo -u {{ username.value }} ssh\" --rsync-path=\"sudo rsync\" {{ option }} {{ b_hostname.value }}:{{ b_path.value }}{} {{ a_path.value }}{} > /tmp/result.log
tail /tmp/result.log

Reference

rsync Filter Rules

Search FILTER RULES in man page of rsync.