Rclone copy md5 error

What is the problem you are having with rclone?

When I try run copy from googledrive to my synology without --ignore-checksum there is following error:

Error

2022/04/17 07:04:50 ERROR : Full Backup 17-04-2022 06:57 / 2022.4.5.tar: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /backupallgemein/ha-backups/Full\\ Backup\\ 17-04-2022\\ 06:57\\ /\\ \\ 2022.4.5.tar": md5sum: '/backupallgemein/ha-backups/Full Backup 17-04-2022 06:57 '$'\357\274\217'' 2022.4.5.tar': No such file or directory: Process exited with status 1 2022/04/17 07:04:50 ERROR : Full Backup 17-04-2022 06:57 / 2022.4.5.tar: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs "" 2022/04/17 07:04:50 ERROR : Attempt 1/3 failed with 2 errors and: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs "" 2022/04/17 07:05:24 ERROR : Full Backup 17-04-2022 06:57 / 2022.4.5.tar: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /backupallgemein/ha-backups/Full\\ Backup\\ 17-04-2022\\ 06:57\\ /\\ \\ 2022.4.5.tar": md5sum: '/backupallgemein/ha-backups/Full Backup 17-04-2022 06:57 '$'\357\274\217'' 2022.4.5.tar': No such file or directory: Process exited with status 1 2022/04/17 07:05:24 ERROR : Full Backup 17-04-2022 06:57 / 2022.4.5.tar: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs "" 2022/04/17 07:05:24 ERROR : Attempt 2/3 failed with 2 errors and: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs "" 2022/04/17 07:06:04 ERROR : Full Backup 17-04-2022 06:57 / 2022.4.5.tar: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /backupallgemein/ha-backups/Full\\ Backup\\ 17-04-2022\\ 06:57\\ /\\ \\ 2022.4.5.tar": md5sum: '/backupallgemein/ha-backups/Full Backup 17-04-2022 06:57 '$'\357\274\217'' 2022.4.5.tar': No such file or directory: Process exited with status 1 2022/04/17 07:06:04 ERROR : Full Backup 17-04-2022 06:57 / 2022.4.5.tar: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs "" 2022/04/17 07:06:04 ERROR : Attempt 3/3 failed with 2 errors and: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs "" 2022/04/17 07:06:04 Failed to copy with 2 errors: last error was: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs ""

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.0

  • os/version: unknown
  • os/kernel: 4.4.180+ (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.8
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive to Synology local

rclone copy gdrive:'Home Assistant Backups' synology:backupallgemein/ha-backups

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

adaniel@ds920:~$ rclone copy gdrive:'Home Assistant Backups' synology:backupallgemein/ha-backups -vv
2022/04/17 07:08:03 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copy" "gdrive:Home Assistant Backups" "synology:backupallgemein/ha-backups" "-vv"]
2022/04/17 07:08:03 DEBUG : Creating backend with remote "gdrive:Home Assistant Backups"
2022/04/17 07:08:03 DEBUG : Using config file from "/var/services/homes/adaniel/.config/rclone/rclone.conf"
2022/04/17 07:08:03 DEBUG : Google drive root 'Home Assistant Backups': 'root_folder_id = 0ADu2upgQWLu0Uk9PVA' - save this in the config to speed up startup
2022/04/17 07:08:04 DEBUG : Creating backend with remote "synology:backupallgemein/ha-backups"
2022/04/17 07:08:04 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: New connection 192.168.178.200:43456->192.168.178.200:22 to "SSH-2.0-OpenSSH_8.2"
2022/04/17 07:08:04 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Using absolute root directory "/backupallgemein/ha-backups"
2022/04/17 07:08:04 DEBUG : Full Backup 16-04-2022 22:32 /  2022.4.5.tar: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/04/17 07:08:04 DEBUG : Full Backup 2022.3.8 2022-04-04 01:00:00.tar: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/04/17 07:08:04 DEBUG : Full Backup 2022.3.8 2022-04-04 01:00:00.tar: Unchanged skipping
2022/04/17 07:08:04 DEBUG : Full Backup 16-04-2022 22:32 /  2022.4.5.tar: Unchanged skipping
2022/04/17 07:08:04 DEBUG : Full Backup 2022.4.3 2022-04-14 01:00:00.tar: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/04/17 07:08:04 DEBUG : Full Backup 2022.4.3 2022-04-14 01:00:00.tar: Unchanged skipping
2022/04/17 07:08:04 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Waiting for checks to finish
2022/04/17 07:08:04 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Waiting for transfers to finish
2022/04/17 07:08:05 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2022/04/17 07:08:05 DEBUG : pacer: Rate limited, increasing sleep to 1.396241334s
2022/04/17 07:08:05 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2022/04/17 07:08:05 DEBUG : pacer: Rate limited, increasing sleep to 2.829325677s
2022/04/17 07:08:06 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2022/04/17 07:08:06 DEBUG : pacer: Rate limited, increasing sleep to 4.22098519s
2022/04/17 07:08:09 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2022/04/17 07:08:09 DEBUG : pacer: Rate limited, increasing sleep to 8.995831799s
2022/04/17 07:08:13 DEBUG : pacer: Reducing sleep to 0s
2022/04/17 07:08:41 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Running remote command: md5sum /backupallgemein/ha-backups/Full\ Backup\ 17-04-2022\ 06:57\ /\ \ 2022.4.5.tar
2022/04/17 07:08:41 ERROR : Full Backup 17-04-2022 06:57 /  2022.4.5.tar: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /backupallgemein/ha-backups/Full\\ Backup\\ 17-04-2022\\ 06:57\\ /\\ \\ 2022.4.5.tar": md5sum: '/backupallgemein/ha-backups/Full Backup 17-04-2022 06:57 '$'\357\274\217''  2022.4.5.tar': No such file or directory: Process exited with status 1
2022/04/17 07:08:41 ERROR : Full Backup 17-04-2022 06:57 /  2022.4.5.tar: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs ""
2022/04/17 07:08:41 INFO  : Full Backup 17-04-2022 06:57 /  2022.4.5.tar: Removing failed copy
2022/04/17 07:08:41 ERROR : Attempt 1/3 failed with 2 errors and: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs ""
2022/04/17 07:08:42 DEBUG : Full Backup 16-04-2022 22:32 /  2022.4.5.tar: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/04/17 07:08:42 DEBUG : Full Backup 16-04-2022 22:32 /  2022.4.5.tar: Unchanged skipping
2022/04/17 07:08:42 DEBUG : Full Backup 2022.3.8 2022-04-04 01:00:00.tar: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/04/17 07:08:42 DEBUG : Full Backup 2022.3.8 2022-04-04 01:00:00.tar: Unchanged skipping
2022/04/17 07:08:42 DEBUG : Full Backup 2022.4.3 2022-04-14 01:00:00.tar: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/04/17 07:08:42 DEBUG : Full Backup 2022.4.3 2022-04-14 01:00:00.tar: Unchanged skipping
2022/04/17 07:08:42 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Waiting for checks to finish
2022/04/17 07:08:42 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Waiting for transfers to finish
2022/04/17 07:09:04 INFO  :
Transferred:      334.547 MiB / 379.102 MiB, 88%, 6.383 MiB/s, ETA 6s
Checks:                 6 / 6, 100%
Transferred:            0 / 1, 0%
Elapsed time:       1m1.1s
Transferring:
 *  Full Backup 17-04-2022 06:57 /  2022.4.5.tar: 76% /189.551Mi, 6.544Mi/s, 6s

2022/04/17 07:09:10 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Running remote command: md5sum /backupallgemein/ha-backups/Full\ Backup\ 17-04-2022\ 06:57\ /\ \ 2022.4.5.tar
2022/04/17 07:09:10 ERROR : Full Backup 17-04-2022 06:57 /  2022.4.5.tar: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /backupallgemein/ha-backups/Full\\ Backup\\ 17-04-2022\\ 06:57\\ /\\ \\ 2022.4.5.tar": md5sum: '/backupallgemein/ha-backups/Full Backup 17-04-2022 06:57 '$'\357\274\217''  2022.4.5.tar': No such file or directory: Process exited with status 1
2022/04/17 07:09:10 ERROR : Full Backup 17-04-2022 06:57 /  2022.4.5.tar: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs ""
2022/04/17 07:09:10 INFO  : Full Backup 17-04-2022 06:57 /  2022.4.5.tar: Removing failed copy
2022/04/17 07:09:10 ERROR : Attempt 2/3 failed with 2 errors and: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs ""
2022/04/17 07:09:10 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2022/04/17 07:09:10 DEBUG : pacer: Rate limited, increasing sleep to 1.847084367s
2022/04/17 07:09:10 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2022/04/17 07:09:10 DEBUG : pacer: Rate limited, increasing sleep to 2.417436267s
2022/04/17 07:09:12 DEBUG : pacer: Reducing sleep to 0s
2022/04/17 07:09:12 DEBUG : Full Backup 16-04-2022 22:32 /  2022.4.5.tar: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/04/17 07:09:12 DEBUG : Full Backup 16-04-2022 22:32 /  2022.4.5.tar: Unchanged skipping
2022/04/17 07:09:12 DEBUG : Full Backup 2022.3.8 2022-04-04 01:00:00.tar: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/04/17 07:09:12 DEBUG : Full Backup 2022.4.3 2022-04-14 01:00:00.tar: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/04/17 07:09:12 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Waiting for checks to finish
2022/04/17 07:09:12 DEBUG : Full Backup 2022.4.3 2022-04-14 01:00:00.tar: Unchanged skipping
2022/04/17 07:09:12 DEBUG : Full Backup 2022.3.8 2022-04-04 01:00:00.tar: Unchanged skipping
2022/04/17 07:09:12 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Waiting for transfers to finish
2022/04/17 07:09:46 DEBUG : sftp://adaniel@192.168.178.200:22/backupallgemein/ha-backups: Running remote command: md5sum /backupallgemein/ha-backups/Full\ Backup\ 17-04-2022\ 06:57\ /\ \ 2022.4.5.tar
2022/04/17 07:09:46 ERROR : Full Backup 17-04-2022 06:57 /  2022.4.5.tar: Failed to calculate dst hash: failed to calculate md5 hash: failed to run "md5sum /backupallgemein/ha-backups/Full\\ Backup\\ 17-04-2022\\ 06:57\\ /\\ \\ 2022.4.5.tar": md5sum: '/backupallgemein/ha-backups/Full Backup 17-04-2022 06:57 '$'\357\274\217''  2022.4.5.tar': No such file or directory: Process exited with status 1
2022/04/17 07:09:46 ERROR : Full Backup 17-04-2022 06:57 /  2022.4.5.tar: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs ""
2022/04/17 07:09:46 INFO  : Full Backup 17-04-2022 06:57 /  2022.4.5.tar: Removing failed copy
2022/04/17 07:09:46 ERROR : Attempt 3/3 failed with 2 errors and: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs ""
2022/04/17 07:09:46 INFO  :
Transferred:      568.652 MiB / 568.652 MiB, 100%, 5.891 MiB/s, ETA 0s
Errors:                 2 (retrying may help)
Checks:                 9 / 9, 100%
Elapsed time:      1m43.0s

2022/04/17 07:09:46 DEBUG : 15 go routines active
2022/04/17 07:09:46 Failed to copy with 2 errors: last error was: corrupted on transfer: md5 hash differ "db288a8d11ced1d7541a5f7c100d0f9b" vs ""

i believe that this has been discussed a number of times in the forum.

https://forum.rclone.org/t/rclone-sftp-no-hashes-found/26942/10

The solution is to add the path:

rclone copy gdrive:'Home Assistant Backups' synology:backupallgemein/ha-backups --sftp-path-override /volume1/backupallgemein/ha-backups

another option, i run rclone executable on the synbox.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.