Rclone not finishing transfers to idrive

Help and Support
1

rclone, latest version on Rocky linux 8.6.

We have created an account with idrive (S3 compatible cloud storage - IDrive® e2), partly because you can get cheap S3-compatible storage there with a very high transfer rate (uploads almost always run at 800-900 MBit with our GBit WAN connection).

The problem is: According to the progress bar, large uploads over 50 GB run without problems up to 100% and then remain there without completing the process. We've already waited more than 1 hour, nothing happens anymore and the files don't appear in idrive either.

image
image798×26 3.5 KB

image
image727×951 61.7 KB

30 minutes later:

image
image688×926 53 KB

2

hi,

a debug log should provide a clue, as with the rest of the help and support template.

3

Agree, especially a debug log with --dump headers to trace the communication with IDrive.

4

Here is the complete log:

5

Thanks!

My quick and immediate impression is that IDrive doesn't respond to this request (within the last 3 minutes of the log):

2022/09/24 19:30:06 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/09/24 19:30:06 DEBUG : HTTP REQUEST (req 0xc000603000)
2022/09/24 19:30:06 DEBUG : POST /kvm/RAM/rfc0ninefrchi7nfv15otthm78?uploadId=8c8097eb-2e6d-4dda-863e-701665eeafc1 HTTP/1.1
Host: w8p7.fra.idrivee2-33.com
User-Agent: rclone/v1.59.2
Content-Length: 954636
Authorization: XXXX
X-Amz-Content-Sha256: f3fa19dd350a71b7a51067956240b8b6c750a1ca61cd3cb31389c28653078fa0
X-Amz-Date: 20220924T193006Z
Accept-Encoding: gzip

2022/09/24 19:30:06 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

I don't know the S3 API well enough to say exactly what is happening, but I guess rclone tries to "commit" the transfer to iDrive and then never receives the "OK, committed successfully" from IDrive.

Please post the remaining items in the help and support template:

  • output from rclone version
  • output from rclone config show idrive_enc: and rclone config show idrive:

then I will ping somebody with more knowledge.

PS:
It would also be helpful to know if you see the same with this slightly different command (removed a leading / and --fast-list):

rclone move /KVM-backups/RAM.qcow2.zst idrive_enc:kvm/RAM --log-level DEBUG --dump headers --log-file=rclone.log
6

[root@rl8 ~]# rclone version
rclone v1.59.2

  • os/version: rocky 8.6 (64 bit)
  • os/kernel: 4.18.0-372.16.1.el8_6.0.1.x86_64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.18.6
  • go/linking: static
  • go/tags: none

rclone config show idrive::
[idrive]
type = s3
provider = Other
env_auth = false
access_key_id = xxx
secret_access_key = xxx
endpoint = w8p7.fra.idrivee2-33.com
acl = private

rclone config show idrive_enc:
[idrive_enc]
type = crypt
remote = idrive:
filename_encryption = standard
directory_name_encryption = false
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***

7

Thanks, did you see the same with the modified command?

@ncw Can you check my analysis above and share any ideas to proceed?

8

Thank you for continuing to work on this.

And, no, the slightly changed command didn't help either. The messages in the log were also the same.

9

@peter999 Raghava from IDrive; recently we had similar complaints from other user and after observing the patterns, we found an edge case and fixed it. So it should be fine now; please check and let us know if you have any issues.

Thanks,

1 Like
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.