Rclone http2: request body larger than specified content length

Help and Support
1

Hello, I had a problem transferring from one remote bucket to another

2022-05-01 00:16:16 ERROR : 10003.html: Failed to copy: Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10003.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72be73*******576c85%2F20220430%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220430T161616Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=c9*******4": http2: request body larger than specified content length
Transferred: 204.287 KiB / 332.355 MiB, 0%, 25.522 KiB/s, ETA 3h42m6s
Errors: 11 (retrying may help)
Transferred: 0 / 10005, 0%

The command I used was as follows:

rclone sync contabo-all-backup:all-backup/datastore/bucket/bc-cdmana-com/sitemap contabo-all-backup-eu2:all-backup/datastore/bucket/bc-cdmana-com/sitemap -P --s3-no-head

What I can confirm is that my configuration is fine.

[contabo-all-backup]
type = s3
provider = Ceph
access_key_id = 7****5
secret_access_key = fc****ae6
endpoint = https://usc1.contabostorage.com

[contabo-all-backup-eu2]
type = s3
provider = Ceph
access_key_id = 72****5
secret_access_key = fc****e6
endpoint = https://eu2.contabostorage.com

These are two remote buckets，It was a sudden problem, and it worked fine at first. can you help me？

2

This is more detailed information

[root@m4078 20220422]# rclone copy -vvv contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list -P --s3-no-head
2022/05/01 01:31:14 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copy" "-vvv" "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list" "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list" "-P" "--s3-no-head"]
2022/05/01 01:31:14 DEBUG : Creating backend with remote "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/01 01:31:14 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/05/01 01:31:14 DEBUG : contabo-all-backup: detected overridden config - adding "{NBTUO}" suffix to name
2022/05/01 01:31:23 DEBUG : fs cache: renaming cache item "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list" to be canonical "contabo-all-backup{NBTUO}:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/01 01:31:23 DEBUG : Creating backend with remote "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/01 01:31:23 DEBUG : contabo-all-backup-eu2: detected overridden config - adding "{NBTUO}" suffix to name
2022/05/01 01:31:23 DEBUG : fs cache: renaming cache item "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list" to be canonical "contabo-all-backup-eu2{NBTUO}:all-backup/datastore/bucket/hz-gsmany-com/list"
2022-05-01 01:32:07 DEBUG : pacer: low level retry 1/2 (error RequestError: send request failed
caused by: Get "https://usc1.contabostorage.com/all-backup?delimiter=%2F&marker=datastore%2Fbucket%2Fhz-gsmany-com%2Flist%2F3175.html&max-keys=1000&prefix=datastore%2Fbucket%2Fhz-gsmany-com%2Flist%2F": read tcp 5.****:44098->104.****:443: read: connection reset by peer)
2022-05-01 01:32:07 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2022-05-01 01:32:38 DEBUG : pacer: Reducing sleep to 0s
2022-05-01 01:33:46 ERROR : 100.html: Failed to copy: Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/100.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72be****178576c85%2F20220430%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220430T173346Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=d2*****d55f": http2: request body larger than specified content length
3

What this error means is that when rclone did the LIST command, the source told it that the length of the item was 100 bytes (say), but when rclone came to fetch the data, 102 bytes (say) was delivered.

Can you do

rclone copy -vv --dump headers contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list -P --s3-no-head --low-level-retries 1 --retries 1

To show the http transactions. We really want --dump bodies to compare the listing with the headers but that will also put your binary data in the log so it will be a bit of a mess to sort out.

You could also try --disable-http2 to see if it is an http2 specific problem or you might need --s3-disable-http2 not sure.

I suspect this is a bug in the http2 layer of the source, but I don't know for sure!

4

I tried a new command with your help，But it's the same mistake. I don't understand why the bytes are inconsistent

[root@m4078 data]# rclone copy -vvv contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list -P --s3-no-head --s3-disable-http2 
2022/05/02 00:56:13 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copy" "-vvv" "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list" "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list" "-P" "--s3-no-head" "--s3-disable-http2"]
2022/05/02 00:56:13 DEBUG : Creating backend with remote "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/02 00:56:13 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/05/02 00:56:13 DEBUG : contabo-all-backup: detected overridden config - adding "{nbNGe}" suffix to name
2022/05/02 00:56:14 DEBUG : fs cache: renaming cache item "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list" to be canonical "contabo-all-backup{nbNGe}:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/02 00:56:14 DEBUG : Creating backend with remote "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/02 00:56:14 DEBUG : contabo-all-backup-eu2: detected overridden config - adding "{nbNGe}" suffix to name
2022/05/02 00:56:14 DEBUG : fs cache: renaming cache item "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list" to be canonical "contabo-all-backup-eu2{nbNGe}:all-backup/datastore/bucket/hz-gsmany-com/list"
2022-05-02 00:56:23 DEBUG : pacer: low level retry 1/1 (error Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72****576c85%2F20220501%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220501T165623Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=ae****db": net/http: HTTP/1.x transport connection broken: http: ContentLength=35325 with Body length 35556)
2022-05-02 00:56:23 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2022-05-02 00:56:23 DEBUG : 10.html: Received error: Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72****576c85%2F20220501%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220501T165623Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=ae****b": net/http: HTTP/1.x transport connection broken: http: ContentLength=35325 with Body length 35556 - low level retry 1/10
2022-05-02 00:56:23 DEBUG : pacer: low level retry 1/1 (error Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72****85%2F20220501%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220501T165623Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=ae****db": net/http: HTTP/1.x transport connection broken: http: ContentLength=35325 with Body length 35556)
2022-05-02 00:56:23 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2022-05-02 00:56:23 DEBUG : 10.html: Received error: Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=7****85%2F20220501%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220501T165623Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=ae****db": net/http: HTTP/1.x transport connection broken: http: ContentLength=35325 with Body length 35556 - low level retry 2/10
5

I carried out the order

rclone copy -vv --dump headers contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list -P --s3-no-head --low-level-retries 1 --retries 1

[root@m4078 data]# rclone copy -vv --dump headers contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list -P --s3-no-head --low-level-retries 1 --retries 1
2022/05/02 01:02:34 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copy" "-vv" "--dump" "headers" "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list" "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list" "-P" "--s3-no-head" "--low-level-retries" "1" "--retries" "1"]
2022/05/02 01:02:34 DEBUG : Creating backend with remote "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/02 01:02:34 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/05/02 01:02:34 DEBUG : contabo-all-backup: detected overridden config - adding "{NBTUO}" suffix to name
2022/05/02 01:02:34 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/05/02 01:02:34 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/05/02 01:02:34 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/05/02 01:02:34 DEBUG : HTTP REQUEST (req 0xc00080dd00)
2022/05/02 01:02:34 DEBUG : HEAD /all-backup/datastore/bucket/hz-gsmany-com/list HTTP/1.1
Host: usc1.contabostorage.com
User-Agent: rclone/v1.58.0
Authorization: XXXX
X-Amz-Content-Sha256: e3******55
X-Amz-Date: 20220501T170234Z

2022/05/02 01:02:34 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2022/05/02 01:02:35 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/05/02 01:02:35 DEBUG : HTTP RESPONSE (req 0xc00080dd00)
2022/05/02 01:02:35 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 219
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Date,Etag,Content-Length,Accept-Ranges,Content-Range,Server,Location,X-Amz-Version-Id
Cf-Cache-Status: DYNAMIC
Cf-Ray: 704a15ed880b9b88-FRA
Content-Type: application/xml
Date: Sun, 01 May 2022 17:02:35 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Ratelimit-Limit: 250
Ratelimit-Remaining: 180
Ratelimit-Reset: 1
Server: cloudflare
Vary: Origin
X-Amz-Request-Id: tx0000048964bb564311404-00626ebd2b-1c214-default
X-Ratelimit-Limit-Second: 250
X-Ratelimit-Remaining-Second: 180

2022/05/02 01:02:35 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/05/02 01:02:35 DEBUG : fs cache: renaming cache item "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list" to be canonical "contabo-all-backup{NBTUO}:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/02 01:02:35 DEBUG : Creating backend with remote "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/02 01:02:35 DEBUG : contabo-all-backup-eu2: detected overridden config - adding "{NBTUO}" suffix to name
2022/05/02 01:02:35 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/05/02 01:02:35 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/05/02 01:02:35 DEBUG : HTTP REQUEST (req 0xc00022ea00)
2022/05/02 01:02:35 DEBUG : HEAD /all-backup/datastore/bucket/hz-gsmany-com/list HTTP/1.1
Host: eu2.contabostorage.com
User-Agent: rclone/v1.58.0
Authorization: XXXX
X-Amz-Content-Sha256: e3******55
X-Amz-Date: 20220501T170235Z

2022/05/02 01:02:35 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/05/02 01:02:35 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/05/02 01:02:35 DEBUG : HTTP RESPONSE (req 0xc00022ea00)
2022/05/02 01:02:35 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 219
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Date,Etag,Content-Length,Accept-Ranges,Content-Range,Server,Location,X-Amz-Version-Id
Cf-Cache-Status: DYNAMIC
Cf-Ray: 704a15f05a306d7f-MUC
Content-Type: application/xml
Date: Sun, 01 May 2022 17:02:35 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Ratelimit-Limit: 250
Ratelimit-Remaining: 249
Ratelimit-Reset: 1
Server: cloudflare
Vary: Origin
X-Amz-Request-Id: tx0000074d03790d0a102fa-00626ebd2b-5f149-default
X-Ratelimit-Limit-Second: 250
X-Ratelimit-Remaining-Second: 249

2022/05/02 01:02:35 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/05/02 01:02:35 DEBUG : fs cache: renaming cache item "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list" to be canonical "contabo-all-backup-eu2{NBTUO}:all-backup/datastore/bucket/hz-gsmany-com/list"
2022-05-02 01:02:35 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022-05-02 01:02:35 DEBUG : HTTP REQUEST (req 0xc00022ec00)
2022-05-02 01:02:35 DEBUG : GET /all-backup?delimiter=%2F&max-keys=1000&prefix=datastore%2Fbucket%2Fhz-gsmany-com%2Flist%2F HTTP/1.1
Host: eu2.contabostorage.com
User-Agent: rclone/v1.58.0
Authorization: XXXX
X-Amz-Content-Sha256: e3******55
X-Amz-Date: 20220501T170235Z
Accept-Encoding: gzip
2022-05-02 01:02:35 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022-05-02 01:02:35 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022-05-02 01:02:35 DEBUG : HTTP REQUEST (req 0xc00080d500)
2022-05-02 01:02:35 DEBUG : GET /all-backup?delimiter=%2F&max-keys=1000&prefix=datastore%2Fbucket%2Fhz-gsmany-com%2Flist%2F HTTP/1.1
Host: usc1.contabostorage.com
User-Agent: rclone/v1.58.0
Authorization: XXXX
X-Amz-Content-Sha256: e3******55
X-Amz-Date: 20220501T170235Z
Accept-Encoding: gzip
2022-05-02 01:02:35 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022-05-02 01:02:35 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022-05-02 01:02:35 DEBUG : HTTP RESPONSE (req 0xc00022ec00)
2022-05-02 01:02:35 DEBUG : HTTP/2.0 200 OK
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Date,Etag,Content-Length,Accept-Ranges,Content-Range,Server,Location,X-Amz-Version-Id
Cf-Cache-Status: DYNAMIC
Cf-Ray: 704a15f0bb0c6d7f-MUC
Content-Type: application/xml
Date: Sun, 01 May 2022 17:02:35 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Ratelimit-Limit: 250
Ratelimit-Remaining: 248
Ratelimit-Reset: 1
Server: cloudflare
Vary: Origin
X-Amz-Request-Id: tx00000c25ad0e12726dbe2-00626ebd2b-57b9e-default
X-Ratelimit-Limit-Second: 250
X-Ratelimit-Remaining-Second: 248
2022-05-02 01:02:35 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022-05-02 01:02:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022-05-02 01:02:36 DEBUG : HTTP RESPONSE (req 0xc00080d500)
2022-05-02 01:02:36 DEBUG : HTTP/2.0 200 OK
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Date,Etag,Content-Length,Accept-Ranges,Content-Range,Server,Location,X-Amz-Version-Id
Cf-Cache-Status: DYNAMIC
Cf-Ray: 704a15f0bfb79b88-FRA
Content-Type: application/xml
Date: Sun, 01 May 2022 17:02:36 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Ratelimit-Limit: 250
Ratelimit-Remaining: 133
Ratelimit-Reset: 1
Server: cloudflare
Vary: Origin
X-Amz-Request-Id: tx00000e74d6183f5487b8c-00626ebd2b-1c214-default
X-Ratelimit-Limit-Second: 250
X-Ratelimit-Remaining-Second: 133
2022-05-02 01:02:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022-05-02 01:02:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022-05-02 01:02:36 DEBUG : HTTP REQUEST (req 0xc00080db00)
2022-05-02 01:02:36 DEBUG : GET /all-backup?delimiter=%2F&marker=datastore%2Fbucket%2Fhz-gsmany-com%2Flist%2F10898.html&max-keys=1000&prefix=datastore%2Fbucket%2Fhz-gsmany-com%2Flist%2F HTTP/1.1
Host: usc1.contabostorage.com
User-Agent: rclone/v1.58.0
Authorization: XXXX
X-Amz-Content-Sha256: e3******55
X-Amz-Date: 20220501T170236Z
Accept-Encoding: gzip
2022-05-02 01:02:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022-05-02 01:02:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022-05-02 01:02:36 DEBUG : HTTP RESPONSE (req 0xc00080db00)
2022-05-02 01:02:36 DEBUG : HTTP/2.0 200 OK
Access-Control-Allow-Origin: *
6

I uploaded an HTML file, and then in the log: accept-encoding: gzip. I don't know why. How should I solve it

7

I tried the --no-gzip-encoding command, but it was the same

[root@m4078 ~]# rclone copy -vvv contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list -P --s3-no-head --s3-disable-http2 --no-gzip-encoding
2022/05/02 01:32:40 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copy" "-vvv" "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list" "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list" "-P" "--s3-no-head" "--s3-disable-http2" "--no-gzip-encoding"]
2022/05/02 01:32:40 DEBUG : Creating backend with remote "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/02 01:32:40 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/05/02 01:32:40 DEBUG : contabo-all-backup: detected overridden config - adding "{nbNGe}" suffix to name

2022/05/02 01:32:40 DEBUG : fs cache: renaming cache item "contabo-all-backup:all-backup/datastore/bucket/hz-gsmany-com/list" to be canonical "contabo-all-backup{nbNGe}:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/02 01:32:40 DEBUG : Creating backend with remote "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list"
2022/05/02 01:32:40 DEBUG : contabo-all-backup-eu2: detected overridden config - adding "{nbNGe}" suffix to name
2022/05/02 01:32:40 DEBUG : fs cache: renaming cache item "contabo-all-backup-eu2:all-backup/datastore/bucket/hz-gsmany-com/list" to be canonical "contabo-all-backup-eu2{nbNGe}:all-backup/datastore/bucket/hz-gsmany-com/list"
2022-05-02 01:32:49 DEBUG : pacer: low level retry 1/1 (error Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10005.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72be731f26a84ec5acdcd94178576c85%2F20220501%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220501T173249Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=8412***8e2a3d7": net/http: HTTP/1.x transport connection broken: http: ContentLength=36117 with Body length 36348)
2022-05-02 01:32:49 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2022-05-02 01:32:49 DEBUG : 10005.html: Received error: Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10005.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72be731f26a84ec5acdcd94178576c85%2F20220501%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220501T173249Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=84***8e2a3d7": net/http: HTTP/1.x transport connection broken: http: ContentLength=36117 with Body length 36348 - low level retry 1/10
2022-05-02 01:32:50 DEBUG : pacer: low level retry 1/1 (error Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10005.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72be731f26a84ec5acdcd94178576c85%2F20220501%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220501T173250Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=9622987***8f1e5144be7": net/http: HTTP/1.x transport connection broken: http: ContentLength=36117 with Body length 36348)
2022-05-02 01:32:50 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2022-05-02 01:32:50 DEBUG : 10005.html: Received error: Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10005.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72be731f26a84ec5acdcd94178576c85%2F20220501%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220501T173250Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=9622***844be7": net/http: HTTP/1.x transport connection broken: http: ContentLength=36117 with Body length 36348 - low level retry 2/10
2022-05-02 01:32:50 DEBUG : pacer: low level retry 1/1 (error Put "https://eu2.contabostorage.com/all-backup/datastore/bucket/hz-gsmany-com/list/10006.html?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=72be731f26a84ec5acdcd94178576c85%2F20220501%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220501T173250Z&X-Amz-Expires=900&X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-acl%3Bx-amz-meta-mtime&X-Amz-Signature=cb****836": net/http: HTTP/1.x transport connection broken: http: ContentLength=35750 with Body length 35981)
2022-05-02 01:32:50 DEBUG : pacer: Rate limited, increasing sleep to 40ms
8

It works fine when I upload other non-HTML files.

2022-05-02 01:35:27 DEBUG : 20220501/dataBackup.tar.gz: multipart upload starting chunk 422 size 6Mi offset 2.467Gi/49.999Gi
2022-05-02 01:35:28 DEBUG : 20220501/dataBackup.tar.gz: multipart upload starting chunk 423 size 6Mi offset 2.473Gi/49.999Gi
2022-05-02 01:35:28 DEBUG : 20220501/dataBackup.tar.gz: multipart upload starting chunk 424 size 6Mi offset 2.479Gi/49.999Gi
2022-05-02 01:35:29 DEBUG : 20220501/dataBackup.tar.gz: multipart upload starting chunk 425 size 6Mi offset 2.484Gi/49.999Gi
2022-05-02 01:35:30 DEBUG : 20220501/dataBackup.tar.gz: multipart upload starting chunk 426 size 6Mi offset 2.490Gi/49.999Gi
2022-05-02 01:35:30 DEBUG : 20220501/dataBackup.tar.gz: multipart upload starting chunk 427 size 6Mi offset 2.496Gi/49.999Gi
2022-05-02 01:35:31 DEBUG : 20220501/dataBackup.tar.gz: multipart upload starting chunk 428 size 6Mi offset 2.502Gi/49.999Gi
2022-05-02 01:35:32 DEBUG : 20220501/dataBackup.tar.gz: multipart upload starting chunk 429 size 6Mi offset 2.508Gi/49.999Gi
2022-05-02 01:35:32 DEBUG : 20220501/dataBackup.tar.gz: multipart upload starting chunk 430 size 6Mi offset 2.514Gi/49.999Gi
Transferred:        2.523 GiB / 49.999 GiB, 5%, 9.642 MiB/s, ETA 1h24m2s
Checks:                 2 / 2, 100%
Transferred:            0 / 1, 0%
Elapsed time:      4m31.1s
Transferring:
 *                    20220501/dataBackup.tar.gz:  5% /49.999Gi, 9.605Mi/s, 1h24m21s
9

Are there any other file types it goes wrong with?

I wonder if the HTML was adjusted by something.

Are you using a proxy that might have altered the data?

Otherwise I suspect a bug in the provider.

10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.