What is the problem you are having with rclone?
I am trying a UNION remote with two "storj s3 backends". Copying a file to union works fine, but copying nested folders leads to an error "Failed to copy: object not found"
Run the command 'rclone version' and share the full output of the command.
# rclone --version
rclone v1.59.0-beta.6139.8e5e230b8
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.13.0-1021-oracle (aarch64)
- os/type: linux
- os/arch: arm64
- go/version: go1.18.2
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Storj s3 backend
The command you were trying to run (eg rclone copy /tmp remote:tmp)
# tree testfolder
testfolder
└── subfolder
└── testfile
1 directory, 1 file
# rclone copy testfolder s3:
2022/05/13 15:22:51 ERROR : subfolder/testfile: Failed to copy: object not found
2022/05/13 15:22:51 ERROR : Attempt 1/3 failed with 1 errors and: object not found
2022/05/13 15:22:52 ERROR : subfolder/testfile: Failed to copy: object not found
2022/05/13 15:22:52 ERROR : Attempt 2/3 failed with 1 errors and: object not found
2022/05/13 15:22:54 ERROR : subfolder/testfile: Failed to copy: object not found
2022/05/13 15:22:54 ERROR : Attempt 3/3 failed with 1 errors and: object not found
2022/05/13 15:22:54 Failed to copy: object not found
The rclone config contents with secrets removed.
[s2]
type = s3
provider = Storj
access_key_id = REDACTED
secret_access_key = REDACTED
endpoint = https://gateway.ap1.storjshare.io
[s3]
type = union
upstreams = s2:whyred s1:whyred
create_policy = eprand
[s1]
type = s3
provider = Storj
env_auth = false
access_key_id = REDACTED
secret_access_key = REDACTED
endpoint = https://gateway.ap1.storjshare.io
whyred is the bucket name in both s1: and s2:
A log from the command with the -vv flag
2022/05/13 15:08:29 DEBUG : rclone: Version "v1.59.0-beta.6139.8e5e230b8" starting with parameters ["rclone" "copy" "testfolder" "s3:" "--log-file" "log.txt" "-vv"]
2022/05/13 15:08:29 DEBUG : Creating backend with remote "testfolder"
2022/05/13 15:08:29 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2022/05/13 15:08:29 DEBUG : fs cache: renaming cache item "testfolder" to be canonical "/root/testfolder"
2022/05/13 15:08:29 DEBUG : Creating backend with remote "s3:"
2022/05/13 15:08:29 DEBUG : Creating backend with remote "s2:whyred"
2022/05/13 15:08:29 DEBUG : Creating backend with remote "s1:whyred"
2022/05/13 15:08:29 DEBUG : union root '': actionPolicy = *policy.EpAll, createPolicy = *policy.EpRand, searchPolicy = *policy.FF
2022/05/13 15:08:30 DEBUG : union root '': Waiting for checks to finish
2022/05/13 15:08:30 DEBUG : union root '': Waiting for transfers to finish
2022/05/13 15:08:31 ERROR : subfolder/testfile: Failed to copy: object not found
2022/05/13 15:08:31 ERROR : Attempt 1/3 failed with 1 errors and: object not found
2022/05/13 15:08:31 DEBUG : union root '': Waiting for checks to finish
2022/05/13 15:08:31 DEBUG : union root '': Waiting for transfers to finish
2022/05/13 15:08:32 ERROR : subfolder/testfile: Failed to copy: object not found
2022/05/13 15:08:32 ERROR : Attempt 2/3 failed with 1 errors and: object not found
2022/05/13 15:08:32 DEBUG : union root '': Waiting for checks to finish
2022/05/13 15:08:32 DEBUG : union root '': Waiting for transfers to finish
2022/05/13 15:08:33 ERROR : subfolder/testfile: Failed to copy: object not found
2022/05/13 15:08:33 ERROR : Attempt 3/3 failed with 1 errors and: object not found
2022/05/13 15:08:33 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 4.0s
2022/05/13 15:08:33 DEBUG : 7 go routines active
2022/05/13 15:08:33 Failed to copy: object not found