What is the problem you are having with rclone?
I'm trying the --max-duration flag, about which the documentation states:
Rclone won't exit with an error if the transfer limit is reached.
However, that doesn't seem to be the case. With a sftp-backed remote:
$ rclone sync --links --create-empty-src-dirs -P --max-duration=19s /volume1/backups hercules:backups --exclude '/@eaDir/**'
2022-08-11 09:20:01 ERROR : borg/data/0/22: Failed to copy: Update ReadFrom failed: context deadline exceeded
2022-08-11 09:20:01 ERROR : borg/data/0/20: Failed to copy: Update ReadFrom failed: context deadline exceeded
2022-08-11 09:20:01 ERROR : borg/data/0/21: Failed to copy: Update ReadFrom failed: context deadline exceeded
2022-08-11 09:20:01 ERROR : borg/data/0/2: Failed to copy: Update ReadFrom failed: context deadline exceeded
2022-08-11 09:20:03 ERROR : sftp://hydra@hercules:22/backups: not deleting files as there were IO errors
2022-08-11 09:20:03 ERROR : sftp://hydra@hercules:22/backups: not deleting directories as there were IO errors
2022-08-11 09:20:03 ERROR : sftp://hydra@hercules:22/backups: max transfer duration reached as set by --max-duration
2022-08-11 09:20:03 ERROR : Fatal error received - not attempting retries
Transferred: 3.979 GiB / 49.685 GiB, 8%, 33.762 MiB/s, ETA 23m6s
Errors: 5 (fatal error encountered)
Checks: 43 / 43, 100%
Transferred: 13 / 10021, 0%
Elapsed time: 1m52.0s
2022/08/11 09:20:03 Failed to sync with 5 errors: last error was: max transfer duration reached as set by --max-duration
The exit code here is 7, again according to the docs indicating a Fatal error.
Run the command 'rclone version' and share the full output of the command.
rclone v1.59.1
- os/version: unknown
- os/kernel: 4.4.180+ (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.5
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
SFTP to a Synology NAS.
The rclone config contents with secrets removed.
[hercules]
type = sftp
host = hercules
user = hydra
key_file = /root/.ssh/id_rsa
shell_type = unix
md5sum_command = none
sha1sum_command = none
A log from the command with the -vv flag
[snip]
2022-08-11 09:33:00 DEBUG : sftp://hydra@hercules:22/backups: Waiting for checks to finish
2022-08-11 09:33:00 DEBUG : sftp://hydra@hercules:22/backups: Waiting for transfers to finish
2022-08-11 09:33:01 DEBUG : borg/name/data/1/1072: Removed after failed upload: context deadline exceeded
2022-08-11 09:33:01 ERROR : borg/name/data/1/1072: Failed to copy: Update ReadFrom failed: context deadline exceeded
2022-08-11 09:33:01 DEBUG : borg/name/data/1/1042: Removed after failed upload: context deadline exceeded
2022-08-11 09:33:01 DEBUG : borg/name/data/1/1071: Removed after failed upload: context deadline exceeded
2022-08-11 09:33:01 ERROR : borg/name/data/1/1042: Failed to copy: Update ReadFrom failed: context deadline exceeded
2022-08-11 09:33:01 ERROR : borg/name/data/1/1071: Failed to copy: Update ReadFrom failed: context deadline exceeded
2022-08-11 09:33:01 DEBUG : borg/name/data/1/1070: Removed after failed upload: context deadline exceeded
2022-08-11 09:33:01 ERROR : borg/name/data/1/1070: Failed to copy: Update ReadFrom failed: context deadline exceeded
2022-08-11 09:33:01 DEBUG : borg/server/data/23: Making directory
2022-08-11 09:33:01 DEBUG : borg/server/data/22: Making directory
2022-08-11 09:33:01 DEBUG : borg/server/data/24: Making directory
2022-08-11 09:33:01 DEBUG : borg/server/data/25: Making directory
2022-08-11 09:33:01 DEBUG : borg/server/data/20: Making directory
2022-08-11 09:33:01 DEBUG : borg/server/data/21: Making directory
2022-08-11 09:33:01 DEBUG : sftp://hydra@hercules:22/backups: failed to copy 4 directories
2022-08-11 09:33:01 DEBUG : sftp://hydra@hercules:22/backups: copied 6 directories
2022-08-11 09:33:01 ERROR : sftp://hydra@hercules:22/backups: not deleting files as there were IO errors
2022-08-11 09:33:01 ERROR : sftp://hydra@hercules:22/backups: not deleting directories as there were IO errors
2022-08-11 09:33:01 ERROR : sftp://hydra@hercules:22/backups: max transfer duration reached as set by --max-duration
2022-08-11 09:33:01 ERROR : Fatal error received - not attempting retries
Transferred: 1.244 GiB / 281.570 GiB, 0%, 35.415 MiB/s, ETA 2h15m5s
Errors: 5 (fatal error encountered)
Checks: 103 / 103, 100%
Transferred: 37 / 10045, 0%
Elapsed time: 38.1s
2022/08/11 09:33:01 INFO :
Transferred: 1.244 GiB / 281.570 GiB, 0%, 35.415 MiB/s, ETA 2h15m5s
Errors: 5 (fatal error encountered)
Checks: 103 / 103, 100%
Transferred: 37 / 10045, 0%
Elapsed time: 38.1s
2022/08/11 09:33:01 DEBUG : 111 go routines active
2022/08/11 09:33:01 DEBUG : sftp://hydra@hercules:22/backups: Closing 12 unused connections
2022/08/11 09:33:01 Failed to sync with 5 errors: last error was: max transfer duration reached as set by --max-duration