Transfer progress keeps resetting to 0

Help and Support
1

What is the problem you are having with rclone?

I'm a few days into a large back up. One file in particular is 580Gb in size.
The transfer progress of this file keeps resetting to 0%. Here are two consecutative updates

Transferred:   	    2.750Ti / 3.051 TiByte, 90%, 3.537 MiByte/s, ETA 1d46m22s
Errors:                 1 (retrying may help)
Checks:               382 / 382, 100%
Transferred:        61188 / 71200, 86%
Elapsed time:  145h30m13.0s
Transferring:
 *                PhatShare/no backup/jordan.tgz: 62% /578.007Gi, 1.281Mi/s, 48h17m50s
 * PhatShare/Pictures/202…022-10-01/_DSC0518.ARW: 45% /51.304Mi, 731.500Ki/s, 39s
 * PhatShare/Pictures/201…019-01-09/_DSC8675.ARW: 43% /41.375Mi, 1.012Mi/s, 22s
 * PhatShare/Movies/Timel…1128_191336_00_377.dng: 23% /35.353Mi, 1.165Mi/s, 23s
[cut]
Transferred:   	    2.752Ti / 3.403 TiByte, 81%, 6.429 MiByte/s, ETA 1d5h30m58s
Errors:                 1 (retrying may help)
Checks:               382 / 382, 100%
Transferred:        61232 / 71244, 86%
Elapsed time:  145h35m13.0s
Transferring:
 *                PhatShare/no backup/jordan.tgz:  0% /578.007Gi, 3.310Mi/s, 49h36m34s
 * PhatShare/Pictures/201…uroraHDR2019-edit.tiff: 64% /239.708Mi, 2.055Mi/s, 41s
 * PhatShare/Pictures/202…022-10-01/_DSC0524.ARW: 77% /50.671Mi, 539.858Ki/s, 21s
 * PhatShare/Pictures/201…019-01-09/_DSC8681.ARW: 50% /41.062Mi, 707.291Ki/s, 29s

The transfer progress of jordan.tgz has reset from 62% to 0%. This has happened several times now. There are no errors or warnings reported between these 2 updates. I don't want to kill the backup in case the file is actually uploading.

Run the command 'rclone version' and share the full output of the command.

I'm using the version of rclone bundled with TrueNas.

rclone v1.56.1
- os/version: debian 11.5 (64 bit)
- os/kernel: 5.15.79+truenas (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.8
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

iDrive which is S3 compatible.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone --config /tmp/tmp0ub4imjj -v --stats 1s --filter-from /tmp/tmp76etynum sync /mnt/PhatPool remote:phat-backup/

The rclone config contents with secrets removed.

[remote]
access_key_id = 
secret_access_key = 
max_upload_parts = 10000
endpoint = r1f4.la.idrivee2-16.com
region =
type = s3
folder = /
bucket = phat-backup
fast_list = false
server_side_encryption =

A log from the command with the -vv flag

This is an in progess back which I don't want to kill if it is likely that the 580Gb file will eventually get uploaded.

2

You have a pretty old version of rlcone. You'd want to update it and test it and then share the log file as that is what is needed to troubleshoot.

3

My provider, iDrive, seems to cancel multipart uploads after about 24 hours. rclone doesn't report the failure and just restarts the upload from 0. Not reporting errors is a poor design in my opinion.

4

Even ones that are in progress? That seems like an iDrive bug to me.

Rclone will report a DEBUG probably and then it will retry the upload. You'd see the message with a -vv log. If the retry fails you'll see an ERROR log eventually.

5

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.