I'm trying to copy bucket from one s3 storage host to another. But if I try to use tree command on source bucket or copy/sync command, they can not be finished and cause a large memory consumption.
Run the command 'rclone version' and share the full output of the command.
rclone v1.59.0
os/version: darwin 12.4 (64 bit)
os/kernel: 21.5.0 (arm64)
os/type: darwin
os/arch: arm64
go/version: go1.18.3
go/linking: dynamic
go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Local company's AWS S3 implementation.
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone -vv tree s3:tvisual_accounts
The rclone config contents with secrets removed.
[s3]
type = s3
provider = AWS
access_key_id = *
secret_access_key = *
region = msk
endpoint = https://s3.*.ru
A log from the command with the -vv flag
2022/07/14 19:18:40 DEBUG : rclone: Version "v1.59.0" starting with parameters ["rclone" "-vv" "tree" "s3:tvisual_accounts"]
2022/07/14 19:18:40 DEBUG : Creating backend with remote "s3:tvisual_accounts"
2022/07/14 19:18:40 DEBUG : Using config file from "/Users/*/.config/rclone/rclone.conf"
I know only that it is s3 and as sre team say it implements 90% of AWS s3 interfaces.
Infinite loop I mean that this tree command execute infinitely. It doesn’t finish. But only for this bucket. Another bucket with only one directory inside it copying without problem
Thank you very much:) It really helps with my problem. After changing provider to "Other" it started to work as expected. I've been thinking all the time that if it implements s3 api I should use AWS provider...
