Rclone sync only new files, but always copying all files

I edited rclone.config and changed endpoint = https://eu2.contabostorage.com
created bucket named test
checked: rclone lsd test:
and now there is output from rclone lsd test: command (listed folder)

Ill try to perform rclone sync -P . bkpspace:test and post here upon finish.

:ok_hand::ok_hand::ok_hand:

To be precise it is not folder (even though rclone uses mkdir command to create it) but bucket (container). All S3 providers I know require data to be stored in buckets. S3 object storage as such does not have concept of folder even most create impression that folders exist (using metadata tricks).

Now I think that you were uploading data without existing bucket. What this particular provider does in such case I have no idea. E.g. your test:

rclone copy file.ext bkpspace:file.ext

on AWS would create new bucket named file.ext and copy file file.ext into it.

Your second attempt to copy the same file showed that it already exist. So better check via UI where these tests are and delete them.

Just a thought but your previous tests showed:

2023/05/18 13:09:43 DEBUG : Creating backend with remote "bkpspace:index.html"
2023/05/18 13:09:43 DEBUG : Resolving service "s3" region "us-east-1"

Maybe data ended up in different regions? You have to investigate yourself. Some S3 providers allow to have the same named buckets in diff regions and EU:bucket1 and US:bucket1 are two different containers.

With new bucket created and changed endpoint = https://eu2.contabostorage.com
rclone sync -P . bkpspace:test is now working as intended!
Everything is in order.
Thank you for the help!

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.