目前这个需求已经有人开发了工具,可以直接结合Proxmox VE 内部的备份工具,可以在Backup Log 中看到Rclone 的相关Log,搭配Proxmoe VE 内建CRON 自动备份到各种网路空间,非常方便:
安装Rclone
在Proxmox VE 中以root 身份登入,直接使用指令安装rclone
apt update apt install rclone -y
或到Rclone 的官网下载安装:
初始化 Rclone
请务必仔细跟着设定,或是你自己去研究rclone 的config
# rclone config
2021/10/08 02:03:49 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> gd-backup_crypt # 务必使用这个名字,或是之后改脚本
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
~~
15 / Google Drive
\ "drive"
~~
Storage> 15
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a string value. Press Enter for the default ("").
client_id> 123456789-xxxxxxxxxxxxxx.apps.googleusercontent.com
OAuth Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Scope that rclone should use when requesting access from drive.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / Full access all files, excluding Application Data Folder.
\ "drive"
~~
scope> 1
ID of the root folder
Leave blank normally.
Fill in to access "Computers" folders (see docs), or for rclone to use
a non root folder as its starting point.
Enter a string value. Press Enter for the default ("").
root_folder_id> xxxx_123456789
# 在Google Drive 建一个资料夹,网址最后一串就是root_folder_id
# https://drive.google.com/drive/u/1/folders/xxxx_123456789
Service Account Credentials JSON file path
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
Enter a string value. Press Enter for the default ("").
service_account_file> [Enter]
Edit advanced config?
y) Yes
n) No (default)
y/n> [Enter]
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine
y) Yes (default)
n) No
y/n> n
Verification code
Go to this URL, authenticate then paste the code here.
https://accounts.google.com/o/oauth2/auth?xxxxxxxxxx
# 复制上面网址到浏览器登入帐号,取得验证码填入下方
Enter a string value. Press Enter for the default ("").
config_verification_code> 123454321
2021/10/08 02:16:42 NOTICE: Make sure your Redirect URL is set to "urn:ietf:wg:oauth:2.0:oob" in your custom config.
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No (default)
y/n> [Enter]
--------------------
[gd-backup_crypt]
type = drive
client_id = 123456789-xxxxxxxxxxxxxx.apps.googleusercontent.com
client_secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
scope = drive
root_folder_id = xxxx_123456789
token = {"access_token":"123454321","token_type":"Bearer","refresh_token":"123454321","expiry":"2021-10-08T03:16:42.034218011+08:00"}
team_drive =
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d>
Current remotes:
Name Type
==== ====
gd-backup_crypt drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q安装proxmox-vzbackup-rclone
下载
以root 身分执行,这是我修改过的版本:
apt-get install git cd /root git clone https://github.com/fuyuanli/proxmox-vzbackup-rclone.git git checkout pve7 chmod +x /root/proxmox-vzbackup-rclone/vzbackup-rclone.sh
修改vzbackup-rclone.sh
修改第5 行至你的备份储存路径
1 #!/bin/bash 2 # ./vzbackup-rclone.sh rehydrate YYYY/MM/DD file_name_encrypted.bin 3 4 ############ /START CONFIG 5 dumpdir="/data/pve/dump" # Set this to where your vzdump files are stored
备份保留的份数会依据你在Proxmox VE 的Backup 选项那边来删除,这个脚本只负责上传档案到云端空间
Proxmox VE 7 修正
参考Issue#7中提到的方法修正,否则会上传失败,我Fork 后修改的版本已经修正了:
修改第18 行 tarfile=${TARFILE} 改成 tarfile=${TARGET}
15 if [ ! -z "${3}" ];then
16 CMDARCHIVE=$(echo "/${3}" | sed -e 's/\(.bin\)*$//g')
17 fi
18 tarfile=${TARGET}
19 exten=${tarfile#*.}
20 filename=${tarfile%.*.*}设定vzdump.conf
编辑 /etc/vzdump.conf 第14 行后加入
script:/root/proxmox-vzbackup-rclone/vzbackup-rclone.sh
后存档
1 # vzdump default settings 2 3 #tmpdir: DIR 4 #dumpdir: DIR 5 #storage: STORAGE_ID 6 #mode: snapshot|suspend|stop 7 #bwlimit: KBPS 8 #ionice: PRI 9 #lockwait: MINUTES 10 #stopwait: MINUTES 11 #stdexcludes: BOOLEAN 12 #mailto: ADDRESSLIST 13 #prune-backups: keep-INTERVAL=N[,...] 14 #script: FILENAME 15 script:/root/proxmox-vzbackup-rclone/vzbackup-rclone.sh 16 #exclude-path: PATHLIST 17 #pigz: N
设定完毕后就可以在Proxmox VE 的界面备份了,每次备份都会触发vzbackup-rclone.sh,再透过 rclone 上传,执行log 如下:
INFO: starting new backup job: vzdump 110 --storage local --node home --compress zstd --remove 0 --mode snapshot
INFO: Deleting backups older than 3 days.
INFO: filesystem type on dumpdir is 'zfs' -using /var/tmp/vzdumptmp2067793_110 for temporary files
INFO: Starting Backup of VM 110 (lxc)
INFO: Backup started at 2021-10-08 18:42:05
INFO: status = running
INFO: CT Name: pykms
INFO: including mount point rootfs ('/') in backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: create storage snapshot 'vzdump'
INFO: creating vzdump archive '/var/lib/vz/dump/vzdump-lxc-110-2021_10_08-18_42_05.tar.zst'
INFO: Total bytes written: 962498560 (918MiB, 65MiB/s)
INFO: archive file size: 274MB
INFO: Backing up /var/lib/vz/dump/vzdump-lxc-110-2021_10_08-18_42_05.tar.zst to remote storage
INFO: rcloning /var/lib/vz/dump/rclone/2021/10
INFO: 2021/10/08 18:43:21 INFO :
INFO: Transferred: 44.496Mi / 274.777 MiByte, 16%, 763.432 KiByte/s, ETA 5m8s
INFO: Transferred: 0 / 1, 0%
INFO: Elapsed time: 1m1.3s
INFO: Transferring:
INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 16% /274.777Mi, 763.432Ki/s, 5m8s
INFO: 2021/10/08 18:44:21 INFO :
INFO: Transferred: 88.684Mi / 274.777 MiByte, 32%, 750.874 KiByte/s, ETA 4m13s
INFO: Transferred: 0 / 1, 0%
INFO: Elapsed time: 2m1.3s
INFO: Transferring:
INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 32% /274.777Mi, 750.874Ki/s, 4m13s
INFO: 2021/10/08 18:45:21 INFO :
INFO: Transferred: 132.840Mi / 274.777 MiByte, 48%, 769.235 KiByte/s, ETA 3m8s
INFO: Transferred: 0 / 1, 0%
INFO: Elapsed time: 3m1.3s
INFO: Transferring:
INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 48% /274.777Mi, 769.235Ki/s, 3m8s
INFO: 2021/10/08 18:46:21 INFO :
INFO: Transferred: 177.402Mi / 274.777 MiByte, 65%, 768.902 KiByte/s, ETA 2m9s
INFO: Transferred: 0 / 1, 0%
INFO: Elapsed time: 4m1.3s
INFO: Transferring:
INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 64% /274.777Mi, 768.902Ki/s, 2m9s
INFO: 2021/10/08 18:47:21 INFO :
INFO: Transferred: 218.777Mi / 274.777 MiByte, 80%, 645.420 KiByte/s, ETA 1m28s
INFO: Transferred: 0 / 1, 0%
INFO: Elapsed time: 5m1.3s
INFO: Transferring:
INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 79% /274.777Mi, 645.420Ki/s, 1m28s
INFO: 2021/10/08 18:48:21 INFO :
INFO: Transferred: 262.652Mi / 274.777 MiByte, 96%, 742.687 KiByte/s, ETA 16s
INFO: Transferred: 0 / 1, 0%
INFO: Elapsed time: 6m1.3s
INFO: Transferring:
INFO: * vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: 95% /274.777Mi, 742.687Ki/s, 16s
INFO: 2021/10/08 18:48:40 INFO : vzdump-lxc-110-2021_10_08-18_42_05.tar.zst: Copied (new)
INFO: 2021/10/08 18:48:40 INFO :
INFO: Transferred: 274.777Mi / 274.777 MiByte, 100%, 668.665 KiByte/s, ETA 0s
INFO: Transferred: 1 / 1, 100%
INFO: Elapsed time: 6m20.2s
INFO: cleanup temporary 'vzdump' snapshot
INFO: Finished Backup of VM 110 (00:06:35)
INFO: Backup finished at 2021-10-08 18:48:40
INFO: Backing up main PVE configs
INFO: Tar files
INFO: rcloning /var/tmp/proxmox-cC0xj8WC/pveConfig.2021-10-08.18.48.40.tgz
INFO: 2021/10/08 18:48:43 INFO : pveConfig.2021-10-08.18.48.40.tgz: Copied (new)
INFO: 2021/10/08 18:48:43 INFO : pveConfig.2021-10-08.18.48.40.tgz: Deleted
INFO: 2021/10/08 18:48:43 INFO :
INFO: Transferred: 34.916Ki / 34.916 KiByte, 100%, 34.910 KiByte/s, ETA 0s
INFO: Checks: 2 / 2, 100%
INFO: Deleted: 1 (files), 0 (dirs)
INFO: Renamed: 1
INFO: Transferred: 1 / 1, 100%
INFO: Elapsed time: 3.0s
INFO: Cleaning up
INFO: Backup job finished successfully
TASK OK跑完的档案会在这两个位置
GoogleDrive/Year/Month/vzdump:VM/LXC 备份
GoogleDrive/Year/Month/pveconfig:/etc/pve/备份
档案还原
使用指令
# 拉取所有备份 ~/proxmox-vzbackup-rclone/vzbackup-rclone.sh rehydrate # 拉取指定文件 ~/proxmox-vzbackup-rclone/vzbackup-rclone.sh rehydrate YYYY/MM/DD file_name.tar.zst
会把上面的备份都下载下来,建议未来有需要时再手动下载就好,也可以直接使用rclone或fclone将指定文件下载到pve的dump目录,手动进行还原。
# 这里的lxc是rclone/fclone中配置的备份虚拟磁盘名称,后面指定了日期和文件 fclone --config /root/.config/rclone/rclone.conf --drive-chunk-size=128M copy lxc:/2022/10/14/vzdump-lxc-100-2022_10_14-08_17_43.tar.zst /var/lib/vz/dump -vP --stats=1s --transfers=16 --checkers=16 --check-first
emengweb