Corrupted on transfer while syncing from NextCloud

Help and Support
23

yes, that is why i asked for you to do that.
rclone does not have special handling for .mp3

but yeah, dump the headers, might find the answer there...

24

I found something... there is a debug log that shows Oc-Checksum header with the "wrong" sha1. I am afraid rclone is using it for comparison, but calculating locally the sha1sum:

I don't know if this is the expected behaviour, I am checking on the Nextcloud documentation too.

The output is very large, i am including just the redacted part:

rclone sync remote:/Music/AC_DC/Blow\ Up\ Your\ Video /target/Music/AC_DC/Blow\ Up\ Your\ Video --error ~/error-nextcloud.txt --progress --include="06 Nick Of Time.bin" --include="06 Nick Of Time.mp3" --retries=1 --dump=headers
2024/04/12 19:16:35 NOTICE: Automatically setting -vv as --dump is enabled
2024/04/12 19:16:35 INFO  : Starting bandwidth limiter at 50Mi Byte/s
2024/04/12 19:16:35 INFO  : Starting transaction limiter: max 4 transactions/s with burst 6
2024/04/12 19:16:35 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "sync" "remote:/Music/AC_DC/Blow Up Your Video" "/target/Music/AC_DC/Blow Up Your Video" "--error" "/home/user/error-nextcloud.txt" "--progress" "--include=06 Nick Of Time.bin" "--include=06 Nick Of Time.mp3" "--retries=1" "--dump=headers"]
2024/04/12 19:16:35 DEBUG : Creating backend with remote "remote:/Music/AC_DC/Blow Up Your Video"
2024/04/12 19:16:35 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2024/04/12 19:16:35 DEBUG : found headers:
2024/04/12 19:16:35 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/04/12 19:16:35 DEBUG : Chunks temporary upload directory: https://server/remote.php/dav/uploads/user/
2024/04/12 19:16:35 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/04/12 19:16:35 DEBUG : HTTP REQUEST (req 0x4000297320)
2024/04/12 19:16:35 DEBUG : PROPFIND /remote.php/dav/files/user/Music/AC_DC/Blow%20Up%20Your%20Video HTTP/1.1


2024/04/12 19:16:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/04/12 19:16:36 DEBUG : HTTP REQUEST (req 0x4000560b40)
2024/04/12 19:16:36 DEBUG : GET /remote.php/dav/files/user/Music/AC_DC/Blow%20Up%20Your%20Video/06%20Nick%20Of%20Time.mp3 HTTP/1.1
Host: server
User-Agent: rclone/v1.66.0
Authorization: XXXX
Depth: 0
Referer: https://server/remote.php/dav/files/user/
Accept-Encoding: gzip
2024/04/12 19:16:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/04/12 19:16:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/04/12 19:16:36 DEBUG : HTTP RESPONSE (req 0x4000560b40)
2024/04/12 19:16:36 DEBUG : HTTP/2.0 200 OK
Content-Length: 8649962
Cache-Control: no-store, no-cache, must-revalidate
Content-Disposition: attachment; filename*=UTF-8''06%20Nick%20Of%20Time.mp3; filename="06%20Nick%20Of%20Time.mp3"
Content-Security-Policy: default-src 'none';
Content-Type: audio/mpeg
Date: Fri, 12 Apr 2024 17:16:36 GMT
Etag: "03a5232cc623efb5b286ff3a0ac97453"
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Last-Modified: Sun, 28 Feb 2016 10:27:44 GMT
Oc-Checksum: SHA1:061a4ad4d97814247f56794282d2c01578eaa9fe MD5:f3577bdef0672f864f76ef388d25d9a3 ADLER32:69ada590
Oc-Etag: "03a5232cc623efb5b286ff3a0ac97453"
Pragma: no-cache
Referrer-Policy: no-referrer
Server: nginx/1.25.4
Set-Cookie: **redacted**
Set-Cookie: **redacted**
Set-Cookie: **redacted**
Set-Cookie: **redacted**
Strict-Transport-Security: max-age=15552000; includeSubDomains
X-Content-Type-Options: nosniff
X-Debug-Token: **redacted**
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Request-Id: ykN1a9Hd4PLf6zBmYjn9
X-Robots-Tag: noindex, nofollow
X-Xss-Protection: 1; mode=block
2024/04/12 19:16:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/04/12 19:16:38 DEBUG : 06 Nick Of Time.mp3: sha1 = 061a4ad4d97814247f56794282d2c01578eaa9fe (webdav root 'Music/AC_DC/Blow Up Your Video')
2024/04/12 19:16:38 DEBUG : 06 Nick Of Time.mp3.kuqevog9.partial: sha1 = 26de37113b5145cea61506a3afdc1ceb585e405f (Local file system at /target/Music/AC_DC/Blow Up Your Video)
2024/04/12 19:16:38 ERROR : 06 Nick Of Time.mp3.kuqevog9.partial: corrupted on transfer: sha1 hash differ "061a4ad4d97814247f56794282d2c01578eaa9fe" vs "26de37113b5145cea61506a3afdc1ceb585e405f"
2024/04/12 19:16:38 INFO  : 06 Nick Of Time.mp3.kuqevog9.partial: Removing failed copy
25

so, rclone is using the hash, as given by nextcloud?
what about the md5, is that correct?

the output should not be too large if you copy a single file, --dump=headers --retries=1

all i see is a blank line?

26

Sorry, I updated the post, I hit enter key by mistake in the middle of preparing it.

27

The md5sum is correct acording to running md5sum on the console of the Nextcloud Server:

f3b175fe21506b3894c71cb38aed0343  06 Nick Of Time.mp3```

```Oc-Checksum: SHA1:061a4ad4d97814247f56794282d2c01578eaa9fe MD5:f3577bdef0672f864f76ef388d25d9a3 ADLER32:69ada590
28

ok, to be clear, the issue is with nextcloud, correct?

29

I think so, I have checked in Nextcloud internal tables, and it is storing the "wrong" SHA1 value for the MP3 file:

SHA1:061a4ad4d97814247f56794282d2c01578eaa9fe MD5:f3577bdef0672f864f76ef388d25d9a3 ADLER32:69ada590

Thank you for you help! I will check with Nextcloud team if it is an open issue or if there is a way to update file checksums.

30

sure, i was just making sure this is not a rclone bug.

ok, if you can, post an update.

31

It looks like Nextcloud is not regenerating checksums, at least on the server, they removed the code in NC v15.
So in order to regenerate the checksums I should find another way, but it is too difficult to deal with it for me, since I am not developing for NextCloud.

1 Like
32

FYI, I doubt this is related but it might be worth retrying with the latest beta due to this recent fix:

33

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.