What is the problem you are having with rclone?
rclone moveto (serverside) is not working. I can't really interpret the errors received and don't know how to solve them.
Run the command 'rclone version' and share the full output of the command.
rclone v1.68.2
- os/version: debian 10.13 (64 bit)
- os/kernel: 5.10.0-0.deb10.16-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.3
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
S3 minio
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone moveto remote:/path/to/file remote:/path/to/file_new
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[cloud]
type = s3
provider = Minio
env_auth = false
access_key_id = XXX
secret_access_key = XXX
endpoint = XXX
acl = bucket-owner-full-control
list_chunk = 10000000
upload_cutoff = 100M
chunk_size = 50M
encoding = Slash,InvalidUtf8,Dot,Percent,Ctl
use_multipart_uploads = true
no_head = false
[cloudcrypt]
type = crypt
remote = cloud:/data/personal-files/backup
password = XXX
A log from the command that you were trying to run with the -vv flag
This is the output when the file is small enough so rclone does not use remote multipart copy (~2.5GB):
2024/12/29 23:30:13 DEBUG : rclone: Version "v1.68.2" starting with parameters ["rclone" "moveto" "cloudcrypt:/path/to/file" "cloudcrypt:/path/to/file_new" "-vv"]
2024/12/29 23:30:13 DEBUG : Creating backend with remote "cloudcrypt:/path/to/file"
2024/12/29 23:30:18 DEBUG : Using config file from "/path/to/rclone.conf"
2024/12/29 23:30:18 DEBUG : Creating backend with remote "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2/9ld7ome5godnn96aagg0b29d7ls1vjt8cmb91gjhqq6c31f2pn9g"
2024/12/29 23:30:21 DEBUG : fs cache: adding new entry for parent of "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2/9ld7ome5godnn96aagg0b29d7ls1vjt8cmb91gjhqq6c31f2pn9g", "cloud:data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2"
2024/12/29 23:30:21 DEBUG : fs cache: adding new entry for parent of "cloudcrypt:/path/to/file", "cloudcrypt:/path/to/"
2024/12/29 23:30:21 DEBUG : Creating backend with remote "cloudcrypt:/path/to/"
2024/12/29 23:30:21 DEBUG : Creating backend with remote "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2"
2024/12/29 23:30:21 DEBUG : fs cache: renaming cache item "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2" to be canonical "cloud:data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2"
2024/12/29 23:30:21 DEBUG : fs cache: switching user supplied name "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2" for canonical name "cloud:data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2"
2024/12/29 23:30:22 DEBUG : file: Need to transfer - File not found at Destination
2024/12/29 23:31:21 INFO :
Transferred: 0 B / 2.543 GiB, 0%, 0 B/s, ETA -
Transferred: 0 / 1, 0%
Elapsed time: 1m7.9s
Transferring:
* file: 0% /2.543Gi, 0/s, -
2024/12/29 23:32:15 DEBUG : pacer: low level retry 1/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 1815C3457DC220E0, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error InternalError: We encountered an internal error, please try again.: cause({"Id":"pydio.grpc.data.index.personal","Code":409,"Detail":"Node path already in use","Status":"Conflict"}))
2024/12/29 23:32:15 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2024/12/29 23:32:21 INFO :
Transferred: 0 B / 2.543 GiB, 0%, 0 B/s, ETA -
Transferred: 0 / 1, 0%
Elapsed time: 2m7.9s
Transferring:
* file: 0% /2.543Gi, 0/s, -
2024/12/29 23:33:21 INFO :
Transferred: 0 B / 2.543 GiB, 0%, 0 B/s, ETA -
Transferred: 0 / 1, 0%
Elapsed time: 3m7.9s
Transferring:
* file: 0% /2.543Gi, 0/s, -
2024/12/29 23:34:21 INFO :
Transferred: 0 B / 2.543 GiB, 0%, 0 B/s, ETA -
Transferred: 0 / 1, 0%
Elapsed time: 4m7.9s
Transferring:
* file: 0% /2.543Gi, 0/s, -
2024/12/29 23:34:25 DEBUG : pacer: low level retry 2/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 1815C363C99109C2, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error InternalError: We encountered an internal error, please try again.: cause({"Id":"pydio.grpc.data.index.personal","Code":409,"Detail":"Node path already in use","Status":"Conflict"}))
2024/12/29 23:34:25 DEBUG : pacer: Rate limited, increasing sleep to 20ms
this goes on indefinitely while rclone increases the sleep time.
When the file is big enough that rclone uses multipart serverside copy (Filesize ~7GB), I get the following output:
2024/12/30 00:01:06 DEBUG : rclone: Version "v1.68.2" starting with parameters ["rclone" "moveto" "cloudcrypt:/path/to/file" "cloudcrypt:/path/to/file_new" "-vv"]
2024/12/30 00:01:06 DEBUG : Creating backend with remote "cloudcrypt:/path/to/file"
2024/12/30 00:01:11 DEBUG : Using config file from "/path/to/rclone.conf"
2024/12/30 00:01:11 DEBUG : Creating backend with remote "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2/hpa64qiii42njtqv0du4dfmj9hdavpli0sbjo3uosoirj2eqbq900t0890upgnj4iptm91c9hdspb1p1m6s5piv7vj1uerbnrtrbaufn4dortjeo0pivmspp06iqif2n5aj7aeg6h0kbejgcagqfed2pfc"
2024/12/30 00:01:15 DEBUG : fs cache: adding new entry for parent of "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2/hpa64qiii42njtqv0du4dfmj9hdavpli0sbjo3uosoirj2eqbq900t0890upgnj4iptm91c9hdspb1p1m6s5piv7vj1uerbnrtrbaufn4dortjeo0pivmspp06iqif2n5aj7aeg6h0kbejgcagqfed2pfc", "cloud:data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2"
2024/12/30 00:01:15 DEBUG : fs cache: adding new entry for parent of "cloudcrypt:/path/to/file", "cloudcrypt:/path/to/"
2024/12/30 00:01:15 DEBUG : Creating backend with remote "cloudcrypt:/path/to/"
2024/12/30 00:01:15 DEBUG : Creating backend with remote "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2"
2024/12/30 00:01:16 DEBUG : fs cache: renaming cache item "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2" to be canonical "cloud:data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2"
2024/12/30 00:01:16 DEBUG : fs cache: switching user supplied name "cloud:/data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2" for canonical name "cloud:data/personal-files/backup/bi80ebufe73ad7v0v52mf70qhc/h8t55tmimu59lt3itbd591jkqo/a0pqukkpvft1mi30q2boh1nn5jgt7sppi5ps6majubi0vocg2bm8djvukrvtbt5a189e733al9qe2"
2024/12/30 00:01:16 DEBUG : file: Need to transfer - File not found at Destination
2024/12/30 00:01:18 DEBUG : hpa64qiii42njtqv0du4dfmj9hdavpli0sbjo3uosoirj2eqbq900t0890upgnj4iptm91c9hdspb1p1m6s5piv7vj1uerbnrtrbaufn4dortjeo0pivmspp06iqif2n5aj7aeg6h0kbejgcagqfed2pfc: Starting multipart copy with 2 parts
2024/12/30 00:01:18 DEBUG : hpa64qiii42njtqv0du4dfmj9hdavpli0sbjo3uosoirj2eqbq900t0890upgnj4iptm91c9hdspb1p1m6s5piv7vj1uerbnrtrbaufn4dortjeo0pivmspp06iqif2n5aj7aeg6h0kbejgcagqfed2pfc: Cancelling multipart copy
2024/12/30 00:01:18 ERROR : file: Failed to copy: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DB71F8A688, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented
2024/12/30 00:01:18 ERROR : file: Not deleting source as copy failed: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DB71F8A688, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented
2024/12/30 00:01:18 ERROR : Attempt 1/3 failed with 1 errors and: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DB71F8A688, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented
2024/12/30 00:01:19 DEBUG : file: Need to transfer - File not found at Destination
2024/12/30 00:01:20 DEBUG : hpa64qiii42njtqv0du4dfmj9hdavpli0sbjo3uosoirj2eqbq900t0890upgnj4iptm91c9hdspb1p1m6s5piv7vj1uerbnrtrbaufn4dortjeo0pivmspp06iqif2n5aj7aeg6h0kbejgcagqfed2pfc: Starting multipart copy with 2 parts
2024/12/30 00:01:20 DEBUG : hpa64qiii42njtqv0du4dfmj9hdavpli0sbjo3uosoirj2eqbq900t0890upgnj4iptm91c9hdspb1p1m6s5piv7vj1uerbnrtrbaufn4dortjeo0pivmspp06iqif2n5aj7aeg6h0kbejgcagqfed2pfc: Cancelling multipart copy
2024/12/30 00:01:20 ERROR : file: Failed to copy: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DBE9BC7BD7, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented
2024/12/30 00:01:20 ERROR : file: Not deleting source as copy failed: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DBE9BC7BD7, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented
2024/12/30 00:01:20 ERROR : Attempt 2/3 failed with 1 errors and: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DBE9BC7BD7, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented
2024/12/30 00:01:21 DEBUG : file: Need to transfer - File not found at Destination
2024/12/30 00:01:21 DEBUG : hpa64qiii42njtqv0du4dfmj9hdavpli0sbjo3uosoirj2eqbq900t0890upgnj4iptm91c9hdspb1p1m6s5piv7vj1uerbnrtrbaufn4dortjeo0pivmspp06iqif2n5aj7aeg6h0kbejgcagqfed2pfc: Starting multipart copy with 2 parts
2024/12/30 00:01:22 DEBUG : hpa64qiii42njtqv0du4dfmj9hdavpli0sbjo3uosoirj2eqbq900t0890upgnj4iptm91c9hdspb1p1m6s5piv7vj1uerbnrtrbaufn4dortjeo0pivmspp06iqif2n5aj7aeg6h0kbejgcagqfed2pfc: Cancelling multipart copy
2024/12/30 00:01:22 ERROR : file: Failed to copy: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DC43022D6A, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented
2024/12/30 00:01:22 ERROR : file: Not deleting source as copy failed: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DC43022D6A, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented
2024/12/30 00:01:22 ERROR : Attempt 3/3 failed with 1 errors and: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DC43022D6A, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented
2024/12/30 00:01:22 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 16.4s
2024/12/30 00:01:22 DEBUG : 11 go routines active
2024/12/30 00:01:22 NOTICE: Failed to moveto: operation error S3: UploadPartCopy, https response error StatusCode: 501, RequestID: 1815C4DC43022D6A, HostID: 6aa0dd8f-38cc-4405-85f1-d092263a4827, api error NotImplemented: A header you provided implies functionality that is not implemented