Config file fails

Digitalocean fedora 31 x64_64 virtual machine.
Want to use rclone sync on digitalocean s3-compatible "spaces."
Trying to sync Bucket 1 to Bucket 2.
rclone v1.51.0

  • os/arch: linux/amd64
  • go version: go1.13.6

[root@fedora]# cat /root/.config/rclone/rclone.conf
[chapelle-show-1]
type = s3
provider = DigitalOcean
access_key_id = XXXXXXXXXXXXXXXXXXXXXX
secret_access_key = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
endpoint = nyc3.digitaloceanspaces.com
acl = private

[chapelle-show-2]
type = s3
provider = DigitalOcean
access_key_id = XXXXXXXXXXXXXXXXXXXXXXXXXXXX
secret_access_key = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
endpoint = nyc3.digitaloceanspaces.com
acl = private

Nothing but errors and I cannot figure out whether my sync command syntax is wrong or if my config file is lacking a specific option:

[root@fedora~]# rclone listremotes
chapelle-show-1:
chapelle-show-2:

[root@fedora~]# rclone sync s3://chapelle-show-1/2019/ s3://chapelle-show-2/2020/
2020/04/15 19:34:02 Failed to create file system for "s3://chapelle-show-1/2019/": didn't find section in config file

[root@fedora~]# rclone sync s3://chapelle-show-1 s3://chapelle-show-2
2020/04/15 19:35:23 Failed to create file system for "s3://chapelle-show-1": didn't find section in config file

[root@fedora~]# rclone --config=/root/.config/rclone/rclone.conf sync s3://chapelle-show-1 s3://chapelle-show-2
2020/04/15 19:40:13 Failed to create file system for "s3://chapelle-show-1": didn't find section in config file

Do I need to mount the s3-compatible spaces before I can sync them?
I'm simply stuck and cannot determine what further options I need to include in the rclone.conf file.
Your guidance and suggestions will be much appreciated.
Thanks.

Those are your remote names so you need to use them in the commands.

rclone sync chapelle-show-1: chapelle-show-2:

Attempt 1 -
[root@fedora~]# rclone sync chapelle-show-1: chapelle-show-2:
<>
Attempt 2 -
[root@fedora~]# rclone sync chapelle-show-1:2019/ chapelle-show-2:2020/
2020/04/15 20:05:59 ERROR : : error reading source directory: directory not found
2020/04/15 20:05:59 ERROR : S3 bucket 2020: not deleting files as there were IO errors
2020/04/15 20:05:59 ERROR : S3 bucket 2020: not deleting directories as there were IO errors
2020/04/15 20:05:59 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2020/04/15 20:05:59 ERROR : : error reading source directory: directory not found
2020/04/15 20:05:59 ERROR : S3 bucket 2020: not deleting files as there were IO errors
2020/04/15 20:05:59 ERROR : S3 bucket 2020: not deleting directories as there were IO errors
2020/04/15 20:05:59 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2020/04/15 20:05:59 ERROR : : error reading source directory: directory not found
2020/04/15 20:05:59 ERROR : S3 bucket 2020: not deleting files as there were IO errors
2020/04/15 20:05:59 ERROR : S3 bucket 2020: not deleting directories as there were IO errors
2020/04/15 20:05:59 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
2020/04/15 20:05:59 Failed to sync: directory not found

Attempt 3 -
[root@fedora~]# rclone sync chapelle-show-1:/2019/ chapelle-show-2:/2020/
2020/04/15 20:06:36 ERROR : : error reading source directory: directory not found
2020/04/15 20:06:36 ERROR : S3 bucket 2020: not deleting files as there were IO errors
2020/04/15 20:06:36 ERROR : S3 bucket 2020: not deleting directories as there were IO errors
2020/04/15 20:06:36 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2020/04/15 20:06:36 ERROR : : error reading source directory: directory not found
2020/04/15 20:06:36 ERROR : S3 bucket 2020: not deleting files as there were IO errors
2020/04/15 20:06:36 ERROR : S3 bucket 2020: not deleting directories as there were IO errors
2020/04/15 20:06:36 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2020/04/15 20:06:36 ERROR : : error reading source directory: directory not found
2020/04/15 20:06:36 ERROR : S3 bucket 2020: not deleting files as there were IO errors
2020/04/15 20:06:36 ERROR : S3 bucket 2020: not deleting directories as there were IO errors
2020/04/15 20:06:36 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
2020/04/15 20:06:36 Failed to sync: directory not found

I confirm there's a folder named "2019" in chapelle-show-1.
I confirm there's a folder named "2020" in chapelle-show-2.

Thank you for your response. Standing by for further guidance if you have a moment.

Attempt 4 -
[root@fedora~]# rclone sync chapelle-show-1:/2019 chapelle-show-2:/2020
2020/04/15 20:12:02 ERROR : : error reading source directory: directory not found
2020/04/15 20:12:02 ERROR : S3 bucket 2020: not deleting files as there were IO errors
2020/04/15 20:12:02 ERROR : S3 bucket 2020: not deleting directories as there were IO errors
2020/04/15 20:12:02 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2020/04/15 20:12:02 ERROR : : error reading source directory: directory not found
2020/04/15 20:12:02 ERROR : S3 bucket 2020: not deleting files as there were IO errors
2020/04/15 20:12:02 ERROR : S3 bucket 2020: not deleting directories as there were IO errors
2020/04/15 20:12:02 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2020/04/15 20:12:02 ERROR : : error reading source directory: directory not found
2020/04/15 20:12:02 ERROR : S3 bucket 2020: not deleting files as there were IO errors
2020/04/15 20:12:02 ERROR : S3 bucket 2020: not deleting directories as there were IO errors
2020/04/15 20:12:02 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
2020/04/15 20:12:02 Failed to sync: directory not found

Same errors with slightly different command syntax

Attempt 5 - I removed the folders in each bucket or space, uploaded one pdf file to chapelle-show-1 and tried to use the copy command to copy it to chapelle-show-2. Still no go.

[root@fedora ~]# rclone copy -v chapelle-show-1:Asus-Router-Guide.pdf chapelle-show-2:
2020/04/16 05:20:00 ERROR : : error reading source directory: directory not found
2020/04/16 05:20:00 INFO : S3 root: Waiting for checks to finish
2020/04/16 05:20:00 INFO : S3 root: Waiting for transfers to finish
2020/04/16 05:20:00 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2020/04/16 05:20:00 ERROR : : error reading source directory: directory not found
2020/04/16 05:20:00 INFO : S3 root: Waiting for checks to finish
2020/04/16 05:20:00 INFO : S3 root: Waiting for transfers to finish
2020/04/16 05:20:00 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2020/04/16 05:20:00 ERROR : : error reading source directory: directory not found
2020/04/16 05:20:00 INFO : S3 root: Waiting for checks to finish
2020/04/16 05:20:00 INFO : S3 root: Waiting for transfers to finish
2020/04/16 05:20:00 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
2020/04/16 05:20:00 Failed to copy: directory not found

Attempt 6 - Try to list the one file in the directory. No go.

[root@fedora log]# rclone ls -vv chapelle-show-1:Asus-Router-User-Guide.pdf
2020/04/16 05:50:08 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "ls" "-vv" "chapelle-show-1:Asus-Router-User-Guide.pdf"]
2020/04/16 05:50:08 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2020/04/16 05:50:08 Failed to ls: directory not found

[root@fedora log]# rclone ls -vv chapelle-show-1/Asus-Router-User-Guide.pdf
2020/04/16 05:50:39 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "ls" "-vv" "chapelle-show-1/Asus-Router-User-Guide.pdf"]
2020/04/16 05:50:39 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2020/04/16 05:50:39 ERROR : : error listing: directory not found
2020/04/16 05:50:39 Failed to ls with 2 errors: last error was: directory not found

rclone ls expects a remote as a paramter, not a filename.

also, this command is missing a colon in the name of the remote
rclone ls -vv chapelle-show-1/Asus-Router-User-Guide.pdf

try
rclone ls -vv chapelle-show-1:

as per docs

Usage:
rclone ls remote:path

Well this is quite an embarrassment, but it wouldn't be the first time I've had a fundamental misunderstanding regarding syntax, etc.

In short, I thought it was necessary to name the remote the same name as the bucket so that rclone could find or recognize the bucket on x, y, or z s3-compatible storage service. This is not necessary.

But if you do this, then it's necessary to repeat the remote name and the bucket name and whatever path to the folders and files thereafter.

This now works:

rclone sync chapelle-show-1:chapelle-show-1/2019/ chapelle-show-2:chapelle-show-2/2020/

I basically confused myself with the naming scheme.
Thank you to those who responded and tried to help.

To good health and the good times ahead.

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.