What is the problem you are having with rclone?
Hello! I want to perform daily backup procedures from various webservers to my local drive by using FTP protocol in combination with rclone sync command. I am having issues with one server, where I am making sync of a whole webserver (includes public_html + mails) - I am getting errors with symlinks, even though I've disabled them by using --skip-links command line parameter. Also, I have trouble with syncing mail to destination (FTP says 550 I can only retrieve regular files - I've attached second log below). By inspecting this files using FTP client, I see that they are also symlinks. Is ignoring symlinks even supported by using FTP location as source with syncing? Is there maybe any other alternative to achieve same behavior for backup procedure? Thanks!
Run the command 'rclone version' and share the full output of the command.
rclone v1.67.0
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.15.0-107-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.4
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
FTP + local drive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone sync Webserver1_FTP:/ /backups --transfers 3 --checkers 3 --create-empty-src-dirs --progress --retries-sleep=60s --quiet --skip-links
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[Webserver1_FTP]
type = ftp
host = XXX
user = XXX
pass = XXX
no_check_certificate = true
explicit_tls = false
tls = false
A log from the command that you were trying to run with the -vv flag
Running backup procedure for 'Test' project
Syncing FTP to local drive
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Transferred: 0 / 2, 0%
Elapsed time: 0.6s
Transferring:
* tmp/.s.PGSQL.5432: transferring
* tmp/mysql.sock: transferring
2024/07/01 20:34:52 ERROR : tmp/.s.PGSQL.5432: Failed to copy: failed to open source object: open: 550 Can't open /.cagefs/tmp/.s.PGSQL.5432: No such file or directory
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Transferred: 0 / 2, 0%
Elapsed time: 0.7s
Transferring:
* tmp/.s.PGSQL.5432: transferring
* tmp/mysql.sock: transferring
2024/07/01 20:34:52 ERROR : tmp/mysql.sock: Failed to copy: failed to open source object: open: 550 Can't open /.cagefs/tmp/mysql.sock: No such file or directory
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 2 (retrying may help)
Transferred: 0 / 1, 0%
Elapsed time: 0.8s
Transferring:
* tmp/mysql.sock: transferringTransferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 2 (retrying may help)
Checks: 1 / 1, 100%
Elapsed time: 1.1s
2024/07/01 20:34:53 ERROR : opt/alt/php44/link/conf: Failed to copy: failed to open source object: open: 550 Can't open /.cagefs/opt/alt/php44/link/conf: No such file or directory
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 3 (retrying may help)
Checks: 1 / 1, 100%
Transferred: 0 / 1, 0%
Elapsed time: 1.3s
Transferring:
* opt/alt/php44/link/conf: transferring
2024/07/01 20:34:53 ERROR : opt/alt/php51/link/conf: Failed to copy: failed to open source object: open: 550 Can't open /.cagefs/opt/alt/php51/link/conf: No such file or directory
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 4 (retrying may help)
Checks: 1 / 1, 100%
Transferred: 0 / 2, 0%
Elapsed time: 1.5s
Transferring:
* opt/alt/php51/link/conf: transferring
* opt/alt/php52/link/conf: transferring
2024/07/01 20:34:53 ERROR : opt/alt/php52/link/conf: Failed to copy: failed to open source object: open: 550 Can't open /.cagefs/opt/alt/php52/link/conf: No such file or directory
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 5 (retrying may help)
Checks: 1 / 1, 100%
Transferred: 0 / 2, 0%
Elapsed time: 1.6s
Transferring:
* opt/alt/php52/link/conf: transferring
* opt/alt/php53/link/conf: transferringTransferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 5 (retrying may help)
Checks: 1 / 1, 100%
Transferred: 0 / 1, 0%
Elapsed time: 1.6s
Transferring:
* opt/alt/php53/link/conf: transferring
2024/07/01 20:34:53 ERROR : opt/alt/php53/link/conf: Failed to copy: failed to open source object: open: 550 Can't open /.cagefs/opt/alt/php53/link/conf: No such file or directory
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 6 (retrying may help)
Checks: 1 / 1, 100%
Transferred: 0 / 2, 0%
Elapsed time: 1.7s
Transferring:
* opt/alt/php53/link/conf: transferring
* opt/alt/php54/link/conf: transferring
2024/07/01 20:34:53 ERROR : opt/alt/php54/link/conf: Failed to copy: failed to open source object: open: 550 Can't open /.cagefs/opt/alt/php54/link/conf: No such file or directory
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 7 (retrying may help)
Checks: 1 / 1, 100%
Transferred: 0 / 1, 0%
Elapsed time: 1.8s
Beside errors above, I am also getting following errors which happens when trying to copy mails:
2024/06/30 03:09:15 ERROR : etc/test.com/user.name.rcube.db.latest.noxitig5.partial: corrupted on transfer: sizes differ src(ftp://ftp.test.at:21) 34 vs dst(Local file system at /backups/Test/ftp) 409600
42.462 MiB / 41.853 MiB, 101%, 130.979 KiB/s, ETA - (xfr#46/59)
2024/06/30 03:09:15 ERROR : etc/test.com/user.name.rcube.db.latest.fetegek2.partial: corrupted on transfer: sizes differ src(ftp://ftp.test.at:21) 34 vs dst(Local file system at /backups/Test/ftp) 389120
42.802 MiB / 42.243 MiB, 101%, 130.979 KiB/s, ETA - (xfr#46/58)
2024/06/30 03:09:15 ERROR : etc/test.com/user.name.rcube.db.latest.damaciv2.partial: corrupted on transfer: sizes differ src(ftp://ftp.test.at:21) 37 vs dst(Local file system at /backups/Test/ftp) 376832
42.974 MiB / 42.614 MiB, 101%, 130.979 KiB/s, ETA - (xfr#46/57)
2024/06/30 03:09:15 ERROR : mail/name.law@test_com: Failed to copy: failed to open source object: open: 550 I can only retrieve regular files
42.974 MiB / 42.974 MiB, 100%, 130.979 KiB/s, ETA 0s (xfr#46/56)
2024/06/30 03:09:15 ERROR : mail/.name@test_com: Failed to copy: failed to open source object: open: 550 I can only retrieve regular files
42.974 MiB / 42.974 MiB, 100%, 130.979 KiB/s, ETA 0s (xfr#46/55)
2024/06/30 03:09:16 ERROR : mail/.hello@test_com: Failed to copy: failed to open source object: open: 550 I can only retrieve regular files
42.974 MiB / 42.974 MiB, 100%, 130.979 KiB/s, ETA 0s (xfr#46/54)
42.974 MiB / 42.974 MiB, 100%, 130.979 KiB/s, ETA 0s (xfr#46/53)
2024/06/30 03:09:16 ERROR : mail/.name1@test_com: Failed to copy: failed to open source object: open: 550 I can only retrieve regular files