What is the problem you are having with rclone?
I can not upload to my Contabo Object Storage a 375 MB file.
Run the command 'rclone version' and share the full output of the command.
rclone v1.67.0
os/version: ubuntu 22.04 (64 bit)
os/kernel: 5.15.0-76-generic (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.22.4
go/linking: static
go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Contabo Object Storage (S3)
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone copy /tmp/PinkMiami-pink-20240704221806.sql Contabo-Storage:backups
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[Contabo-Storage]
type = s3
location_constraint = US-central
provider = Other
access_key_id = XXX
secret_access_key = XXX
endpoint = https://usc1.contabostorage.com/backups
A log from the command that you were trying to run with the -vv
flag
2024/07/04 22:45:36 ERROR : PinkMiami-pink-20240704221806.sql: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 4 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx00000d5a187e2e93ddc3c-0066875e50-472cb9-default, host id:
2024/07/04 22:45:36 ERROR : Attempt 1/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 4 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx00000d5a187e2e93ddc3c-0066875e50-472cb9-default, host id:
2024/07/04 22:45:37 ERROR : PinkMiami-pink-20240704221806.sql: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 4 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx0000043ffeb839d7faadb-0066875e51-472c32-default, host id:
2024/07/04 22:45:37 ERROR : Attempt 2/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 4 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx0000043ffeb839d7faadb-0066875e51-472c32-default, host id:
2024/07/04 22:45:38 ERROR : PinkMiami-pink-20240704221806.sql: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 3 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx00000e04ddfe0b016509b-0066875e52-472c1e-default, host id:
2024/07/04 22:45:38 ERROR : Attempt 3/3 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 3 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx00000e04ddfe0b016509b-0066875e52-472c1e-default, host id:
2024/07/04 22:45:38 Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 3 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx00000e04ddfe0b016509b-0066875e52-472c1e-default, host id:
asdffdsa
(jojothehumanmonkey)
July 5, 2024, 3:04am
2
Thanks for your quick response, I made the provider change and these are the results:
root@vmi1347025:~# rclone copy /tmp/PinkMiami-pink-20240704221806.sql Contabo-Storage:backups -vv --retries=1
2024/07/05 11:02:52 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "/tmp/PinkMiami-pink-20240704221806.sql" "Contabo-Storage:backups" "-vv" "--retries=1"]
2024/07/05 11:02:52 DEBUG : Creating backend with remote "/tmp/PinkMiami-pink-20240704221806.sql"
2024/07/05 11:02:52 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/07/05 11:02:52 DEBUG : fs cache: adding new entry for parent of "/tmp/PinkMiami-pink-20240704221806.sql", "/tmp"
2024/07/05 11:02:52 DEBUG : Creating backend with remote "Contabo-Storage:backups"
2024/07/05 11:02:52 DEBUG : Resolving service "s3" region "us-east-1"
2024/07/05 11:02:52 DEBUG : PinkMiami-pink-20240704221806.sql: Need to transfer - File not found at Destination
2024/07/05 11:02:52 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: disabling buffering because source is local disk
2024/07/05 11:02:52 INFO : S3 bucket backups: Bucket "backups" created with ACL ""
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: open chunk writer: started multipart upload: 2~nzHkZpfjhsB4DywCksX6wrGZTlgfoZ_
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 4
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: Starting multi-thread copy with 75 chunks of size 5Mi with 4 parallel streams
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 4/75 (15728640-20971520) size 5Mi starting
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 1/75 (0-5242880) size 5Mi starting
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 2/75 (5242880-10485760) size 5Mi starting
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 3/75 (10485760-15728640) size 5Mi starting
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: Seek from 5242880 to 0
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: Seek from 5242880 to 0
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: Seek from 5242880 to 0
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: Seek from 5242880 to 0
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: Seek from 5242880 to 0
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 1/75 failed: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx000007affdad87af3d27d-0066880b1e-472c14-default, host id:
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 5/75 (20971520-26214400) size 5Mi starting
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 2/75 failed: multi-thread copy: failed to write chunk: failed to upload chunk 2 with 5242880 bytes: RequestCanceled: request context canceled
caused by: context canceled
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 3/75 failed: multi-thread copy: failed to write chunk: failed to upload chunk 3 with 5242880 bytes: RequestCanceled: request context canceled
caused by: context canceled
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 4/75 failed: multi-thread copy: failed to write chunk: failed to upload chunk 4 with 5242880 bytes: RequestCanceled: request context canceled
caused by: context canceled
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: Seek from 5242880 to 0
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: chunk 5/75 failed: multi-thread copy: failed to write chunk: failed to upload chunk 5 with 5242880 bytes: RequestCanceled: request context canceled
caused by: context canceled
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: cancelling transfer on exit
2024/07/05 11:02:54 DEBUG : PinkMiami-pink-20240704221806.sql: multi-thread copy: abort failed: failed to abort multipart upload "2~nzHkZpfjhsB4DywCksX6wrGZTlgfoZ_": NoSuchUpload:
status code: 404, request id: tx00000fc1fa675935854dc-0066880b1e-470a5f-default, host id:
2024/07/05 11:02:54 ERROR : PinkMiami-pink-20240704221806.sql: Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx000007affdad87af3d27d-0066880b1e-472c14-default, host id:
2024/07/05 11:02:54 ERROR : Attempt 1/1 failed with 1 errors and: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx000007affdad87af3d27d-0066880b1e-472c14-default, host id:
2024/07/05 11:02:54 INFO :
Transferred: 864 KiB / 864 KiB, 100%, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 1.8s
2024/07/05 11:02:54 DEBUG : 8 go routines active
2024/07/05 11:02:54 Failed to copy: multi-thread copy: failed to write chunk: failed to upload chunk 1 with 5242880 bytes: NoSuchUpload:
status code: 404, request id: tx000007affdad87af3d27d-0066880b1e-472c14-default, host id:
asdffdsa
(jojothehumanmonkey)
July 5, 2024, 3:19pm
4
perhaps that should be lowercase.
perhaps remove /backups
can you post rclone config redacted Contabo-Storage:
1 Like
As you adviced I removed the /backups at the end of the endpoint and then it uploaded the file. Thanks a lot, this is my first time working with rclone.
system
(system)
Closed
July 8, 2024, 3:41pm
6
This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.