Incorrect Output during copyto retry

SteveO · December 19, 2023, 10:30am

What is the problem you are having with rclone?

Transferred stats are incorrect when an upload fails, then retries.
Test Steps:
1 start copy to remote location
2 during transfer mod the file - e.g. touch
3 rclone copy fails
4 copy retries but the upload stats are incorrect
5 also rclone appears to stall for a period, e.g. in log nothing seems to be happening for 3-4 minutes
6 the copy is ultimately successful, but with the erroneous output it is not possible to track progress accurately

The copy is reporting 100% transfer when it is only partially copied
e.g. in the log below, a file of ~686Mi is being copied, and is modded at ~350Mi. The next stats say 100% Transferred and ETA of 0s, even though only 50% of the file is transferred. Then, after a pause of 3-4 minutes, the transfer restarts and is successful.

The issue seems to be related to how rclone deals with the interruption, and retry. It reports 100% transferred when it is not, and it also appears to stall for a period before restarting the transfer.

This is very intermittent - so it does not happen every time - maybe 1 out of 20-30 attempts to reproduce the error actually reproduce the error.

I have tried to ensure a file is not modified when transfer has started, so hopefully this is less likely to happen, but it still looks like a possible bug.

I will try to get the latest rclone version installed and try again.

rclone version

rclone v1.58.1-DEV

os/version: ubuntu 18.04 (64 bit)
os/kernel: 4.15.0-1080-ibm-gt (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.13.8
go/linking: dynamic
go/tags: none

Which cloud storage system are you using? (eg Google Drive)

IBMCOS

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone -vv --stats=55s --s3-disable-checksum --stats-file-name-length=0 --low-level-retries=5 copyto <source-file> <destination>```


#### The rclone config contents with secrets removed.

[IBMCOS]
type = s3
endpoint = s3.us-south.cloud-object-storage.appdomain.cloud
provider = IBMCOS
access_key_id =
env_auth = false
location_constraint = us-standard
secret_access_key =
region = us-south
acl = private

[IBMCOS-chunker]
type = chunker
remote = IBMCOS:
chunk_size = 1G
fail_hard = true
hash_type = none```

A log from the command with the `-vv` flag

2023/12/19 09:08:52 DEBUG : rclone: Version "v1.58.1-DEV" starting with parameters ["rclone" "-vv" "--stats=55s" "--config=/var/lib/rclone-config/rclone-cos-writer" "--s3-disable-checksum" "--stats-file-name-length=0" "--low-level-retries=5" "copyto" "/tmp/test.qcow2" "IBMCOS:<bucket-name>/test/test.qcow2"]
2023/12/19 09:08:52 DEBUG : Creating backend with remote "/tmp/test.qcow2"
2023/12/19 09:08:52 DEBUG : Using config file from "/var/lib/rclone-config/rclone-cos-writer"
2023/12/19 09:08:52 DEBUG : fs cache: adding new entry for parent of "/tmp/test.qcow2", "/tmp"
2023/12/19 09:08:52 DEBUG : Creating backend with remote "/tmp/test.qcow2"
2023/12/19 09:08:52 DEBUG : Using config file from "/var/lib/rclone-config/rclone-cos-writer"
2023/12/19 09:08:52 DEBUG : fs cache: adding new entry for parent of "/tmp/test.qcow2", "/tmp"
2023/12/19 09:08:52 DEBUG : Creating backend with remote "IBMCOS:<bucket-name>/test/"
2023/12/19 09:08:52 DEBUG : IBMCOS: detected overridden config - adding "{1SSjr}" suffix to name
2023/12/19 09:08:52 DEBUG : fs cache: renaming cache item "IBMCOS:<bucket-name>/test/" to be canonical "IBMCOS{1SSjr}:<bucket-name>/test"
2023/12/19 09:08:52 DEBUG : test.qcow2: Need to transfer - File not found at Destination
2023/12/19 09:08:52 DEBUG : test.qcow2: multipart upload starting chunk 1 size 5Mi offset 0/686.125Mi
2023/12/19 09:08:52 DEBUG : test.qcow2: multipart upload starting chunk 2 size 5Mi offset 5Mi/686.125Mi
2023/12/19 09:08:52 DEBUG : test.qcow2: multipart upload starting chunk 3 size 5Mi offset 10Mi/686.125Mi
2023/12/19 09:08:52 DEBUG : test.qcow2: multipart upload starting chunk 4 size 5Mi offset 15Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 5 size 5Mi offset 20Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 6 size 5Mi offset 25Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 7 size 5Mi offset 30Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 8 size 5Mi offset 35Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 9 size 5Mi offset 40Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 10 size 5Mi offset 45Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 11 size 5Mi offset 50Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 12 size 5Mi offset 55Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 13 size 5Mi offset 60Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 14 size 5Mi offset 65Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 15 size 5Mi offset 70Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 16 size 5Mi offset 75Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 17 size 5Mi offset 80Mi/686.125Mi
2023/12/19 09:08:53 DEBUG : test.qcow2: multipart upload starting chunk 18 size 5Mi offset 85Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 19 size 5Mi offset 90Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 20 size 5Mi offset 95Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 21 size 5Mi offset 100Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 22 size 5Mi offset 105Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 23 size 5Mi offset 110Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 24 size 5Mi offset 115Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 25 size 5Mi offset 120Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 26 size 5Mi offset 125Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 27 size 5Mi offset 130Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 28 size 5Mi offset 135Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 29 size 5Mi offset 140Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 30 size 5Mi offset 145Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 31 size 5Mi offset 150Mi/686.125Mi
2023/12/19 09:08:54 DEBUG : test.qcow2: multipart upload starting chunk 32 size 5Mi offset 155Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 33 size 5Mi offset 160Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 34 size 5Mi offset 165Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 35 size 5Mi offset 170Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 36 size 5Mi offset 175Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 37 size 5Mi offset 180Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 38 size 5Mi offset 185Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 39 size 5Mi offset 190Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 40 size 5Mi offset 195Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 41 size 5Mi offset 200Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 42 size 5Mi offset 205Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 43 size 5Mi offset 210Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 44 size 5Mi offset 215Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 45 size 5Mi offset 220Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 46 size 5Mi offset 225Mi/686.125Mi
2023/12/19 09:08:55 DEBUG : test.qcow2: multipart upload starting chunk 47 size 5Mi offset 230Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 48 size 5Mi offset 235Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 49 size 5Mi offset 240Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 50 size 5Mi offset 245Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 51 size 5Mi offset 250Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 52 size 5Mi offset 255Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 53 size 5Mi offset 260Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 54 size 5Mi offset 265Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 55 size 5Mi offset 270Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 56 size 5Mi offset 275Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 57 size 5Mi offset 280Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 58 size 5Mi offset 285Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 59 size 5Mi offset 290Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 60 size 5Mi offset 295Mi/686.125Mi
2023/12/19 09:08:56 DEBUG : test.qcow2: multipart upload starting chunk 61 size 5Mi offset 300Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: multipart upload starting chunk 62 size 5Mi offset 305Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: multipart upload starting chunk 63 size 5Mi offset 310Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: multipart upload starting chunk 64 size 5Mi offset 315Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: multipart upload starting chunk 65 size 5Mi offset 320Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: multipart upload starting chunk 66 size 5Mi offset 325Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: multipart upload starting chunk 67 size 5Mi offset 330Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: multipart upload starting chunk 68 size 5Mi offset 335Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: multipart upload starting chunk 69 size 5Mi offset 340Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: multipart upload starting chunk 70 size 5Mi offset 345Mi/686.125Mi
2023/12/19 09:08:57 DEBUG : test.qcow2: Cancelling multipart upload
2023/12/19 09:08:57 ERROR : test.qcow2: Failed to copy: multipart upload failed to read source: can't copy - source file is being updated (mod time changed from 2023-12-19 09:06:08.580733226 +0000 UTC to 2023-12-19 09:08:57.504271027 +0000 UTC)
2023/12/19 09:08:57 ERROR : Attempt 1/3 failed with 1 errors and: multipart upload failed to read source: can't copy - source file is being updated (mod time changed from 2023-12-19 09:06:08.580733226 +0000 UTC to 2023-12-19 09:08:57.504271027 +0000 UTC)
2023/12/19 09:08:57 DEBUG : pacer: low level retry 1/2 (error NoSuchUpload: Invalid uploadId:0100018c-8155-cb45-e23c-98c1170567b4
        status code: 404, request id: d6a88aaa-350f-4f1e-81fe-9c8fa83674b8, host id: )
2023/12/19 09:08:57 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2023/12/19 09:08:57 DEBUG : pacer: Reducing sleep to 0s
2023/12/19 09:08:57 DEBUG : pacer: low level retry 2/2 (error NoSuchUpload: Invalid uploadId:0100018c-8155-cb45-e23c-98c1170567b4
        status code: 404, request id: e125a55e-6776-472e-9972-26321f197548, host id: )
2023/12/19 09:08:57 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2023/12/19 09:09:47 INFO  :
Transferred:      350.996 MiB / 350.996 MiB, 100%, 1.888 MiB/s, ETA 0s
Elapsed time:        55.0s

2023/12/19 09:10:42 INFO  :
Transferred:      350.996 MiB / 350.996 MiB, 100%, 950.551 KiB/s, ETA 0s
Elapsed time:      1m50.0s

2023/12/19 09:11:37 INFO  :
Transferred:      350.996 MiB / 350.996 MiB, 100%, 950.551 KiB/s, ETA 0s
Elapsed time:      2m45.0s

2023/12/19 09:12:32 INFO  :
Transferred:      350.996 MiB / 350.996 MiB, 100%, 950.551 KiB/s, ETA 0s
Elapsed time:      3m40.0s

2023/12/19 09:13:27 INFO  :
Transferred:      350.996 MiB / 350.996 MiB, 100%, 950.551 KiB/s, ETA 0s
Elapsed time:      4m35.0s

2023/12/19 09:13:57 DEBUG : pacer: Reducing sleep to 0s
2023/12/19 09:13:57 DEBUG : test.qcow2: Need to transfer - File not found at Destination
2023/12/19 09:13:57 DEBUG : test.qcow2: multipart upload starting chunk 1 size 5Mi offset 0/686.125Mi
2023/12/19 09:13:57 DEBUG : test.qcow2: multipart upload starting chunk 2 size 5Mi offset 5Mi/686.125Mi
2023/12/19 09:13:57 DEBUG : test.qcow2: multipart upload starting chunk 3 size 5Mi offset 10Mi/686.125Mi
2023/12/19 09:13:57 DEBUG : test.qcow2: multipart upload starting chunk 4 size 5Mi offset 15Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 5 size 5Mi offset 20Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 6 size 5Mi offset 25Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 7 size 5Mi offset 30Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 8 size 5Mi offset 35Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 9 size 5Mi offset 40Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 10 size 5Mi offset 45Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 11 size 5Mi offset 50Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 12 size 5Mi offset 55Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 13 size 5Mi offset 60Mi/686.125Mi
2023/12/19 09:13:58 DEBUG : test.qcow2: multipart upload starting chunk 14 size 5Mi offset 65Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 15 size 5Mi offset 70Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 16 size 5Mi offset 75Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 17 size 5Mi offset 80Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 18 size 5Mi offset 85Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 19 size 5Mi offset 90Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 20 size 5Mi offset 95Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 21 size 5Mi offset 100Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 22 size 5Mi offset 105Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 23 size 5Mi offset 110Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 24 size 5Mi offset 115Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 25 size 5Mi offset 120Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 26 size 5Mi offset 125Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 27 size 5Mi offset 130Mi/686.125Mi
2023/12/19 09:13:59 DEBUG : test.qcow2: multipart upload starting chunk 28 size 5Mi offset 135Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 29 size 5Mi offset 140Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 30 size 5Mi offset 145Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 31 size 5Mi offset 150Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 32 size 5Mi offset 155Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 33 size 5Mi offset 160Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 34 size 5Mi offset 165Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 35 size 5Mi offset 170Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 36 size 5Mi offset 175Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 37 size 5Mi offset 180Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 38 size 5Mi offset 185Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 39 size 5Mi offset 190Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 40 size 5Mi offset 195Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 41 size 5Mi offset 200Mi/686.125Mi
2023/12/19 09:14:00 DEBUG : test.qcow2: multipart upload starting chunk 42 size 5Mi offset 205Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 43 size 5Mi offset 210Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 44 size 5Mi offset 215Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 45 size 5Mi offset 220Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 46 size 5Mi offset 225Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 47 size 5Mi offset 230Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 48 size 5Mi offset 235Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 49 size 5Mi offset 240Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 50 size 5Mi offset 245Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 51 size 5Mi offset 250Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 52 size 5Mi offset 255Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 53 size 5Mi offset 260Mi/686.125Mi
2023/12/19 09:14:01 DEBUG : test.qcow2: multipart upload starting chunk 54 size 5Mi offset 265Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 55 size 5Mi offset 270Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 56 size 5Mi offset 275Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 57 size 5Mi offset 280Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 58 size 5Mi offset 285Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 59 size 5Mi offset 290Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 60 size 5Mi offset 295Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 61 size 5Mi offset 300Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 62 size 5Mi offset 305Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 63 size 5Mi offset 310Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 64 size 5Mi offset 315Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 65 size 5Mi offset 320Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 66 size 5Mi offset 325Mi/686.125Mi
2023/12/19 09:14:02 DEBUG : test.qcow2: multipart upload starting chunk 67 size 5Mi offset 330Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 68 size 5Mi offset 335Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 69 size 5Mi offset 340Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 70 size 5Mi offset 345Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 71 size 5Mi offset 350Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 72 size 5Mi offset 355Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 73 size 5Mi offset 360Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 74 size 5Mi offset 365Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 75 size 5Mi offset 370Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 76 size 5Mi offset 375Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 77 size 5Mi offset 380Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 78 size 5Mi offset 385Mi/686.125Mi
2023/12/19 09:14:03 DEBUG : test.qcow2: multipart upload starting chunk 79 size 5Mi offset 390Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 80 size 5Mi offset 395Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 81 size 5Mi offset 400Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 82 size 5Mi offset 405Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 83 size 5Mi offset 410Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 84 size 5Mi offset 415Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 85 size 5Mi offset 420Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 86 size 5Mi offset 425Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 87 size 5Mi offset 430Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 88 size 5Mi offset 435Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 89 size 5Mi offset 440Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 90 size 5Mi offset 445Mi/686.125Mi
2023/12/19 09:14:04 DEBUG : test.qcow2: multipart upload starting chunk 91 size 5Mi offset 450Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 92 size 5Mi offset 455Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 93 size 5Mi offset 460Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 94 size 5Mi offset 465Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 95 size 5Mi offset 470Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 96 size 5Mi offset 475Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 97 size 5Mi offset 480Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 98 size 5Mi offset 485Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 99 size 5Mi offset 490Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 100 size 5Mi offset 495Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 101 size 5Mi offset 500Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 102 size 5Mi offset 505Mi/686.125Mi
2023/12/19 09:14:05 DEBUG : test.qcow2: multipart upload starting chunk 103 size 5Mi offset 510Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 104 size 5Mi offset 515Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 105 size 5Mi offset 520Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 106 size 5Mi offset 525Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 107 size 5Mi offset 530Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 108 size 5Mi offset 535Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 109 size 5Mi offset 540Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 110 size 5Mi offset 545Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 111 size 5Mi offset 550Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 112 size 5Mi offset 555Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 113 size 5Mi offset 560Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 114 size 5Mi offset 565Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 115 size 5Mi offset 570Mi/686.125Mi
2023/12/19 09:14:06 DEBUG : test.qcow2: multipart upload starting chunk 116 size 5Mi offset 575Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 117 size 5Mi offset 580Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 118 size 5Mi offset 585Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 119 size 5Mi offset 590Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 120 size 5Mi offset 595Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 121 size 5Mi offset 600Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 122 size 5Mi offset 605Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 123 size 5Mi offset 610Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 124 size 5Mi offset 615Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 125 size 5Mi offset 620Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 126 size 5Mi offset 625Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 127 size 5Mi offset 630Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 128 size 5Mi offset 635Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 129 size 5Mi offset 640Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 130 size 5Mi offset 645Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 131 size 5Mi offset 650Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 132 size 5Mi offset 655Mi/686.125Mi
2023/12/19 09:14:07 DEBUG : test.qcow2: multipart upload starting chunk 133 size 5Mi offset 660Mi/686.125Mi
2023/12/19 09:14:08 DEBUG : test.qcow2: multipart upload starting chunk 134 size 5Mi offset 665Mi/686.125Mi
2023/12/19 09:14:08 DEBUG : test.qcow2: multipart upload starting chunk 135 size 5Mi offset 670Mi/686.125Mi
2023/12/19 09:14:08 DEBUG : test.qcow2: multipart upload starting chunk 136 size 5Mi offset 675Mi/686.125Mi
2023/12/19 09:14:08 DEBUG : test.qcow2: multipart upload starting chunk 137 size 5Mi offset 680Mi/686.125Mi
2023/12/19 09:14:08 DEBUG : test.qcow2: multipart upload starting chunk 138 size 1.125Mi offset 685Mi/686.125Mi
2023/12/19 09:14:08 INFO  : test.qcow2: Copied (new)
2023/12/19 09:14:08 ERROR : Attempt 2/3 succeeded
2023/12/19 09:14:08 INFO  :
Transferred:        1.013 GiB / 1.013 GiB, 100%, 950.551 KiB/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:      5m16.0s

2023/12/19 09:14:08 DEBUG : 12 go routines active```

kapitainsky · December 19, 2023, 10:38am

A lot of things have changed since your ancient rclone version.... There were 20+ versions released since then... tones of fixes and improvements

If you want to draw some meaningful conclusions please repeat your tests with the latest rclone version. Otherwise I do not think it is worth to even look at this - unless somebody is interested how some historical rclone version worked.

SteveO · December 19, 2023, 10:47am

Thanks Kapitainsky,

I intend to try with a later version, but thought it worthwhile raising the topic.

kapitainsky · December 19, 2023, 10:53am

Why? What's the value in raising topic that you maybe discovered a bug in some very old rclone version? Before testing with the latest version?

Let's see what your new version tests show.

system · January 18, 2024, 10:54am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.