I installed rclone v1.62.2 on a CentOS 7 VM on GCP. This VM has a service account that has the necessary IAM permissions to access the GCS buckets.
We have two configurations
local
GCS
We configured the explorer to have the GCS on the lefthand side and the local on the righthand side.
Some of our copying operations fail. Below, you can find our scenarios:
Upload file from local subfolder to remote bucket: WORKS
Upload file from local subfolder to remote subfolder in a bucket: WORKS
Upload folder from local subfolder to remote bucket: WORKS
Upload folder from local subfolder to remote subfolder in a bucket: FAILS
Local path: local:/subfolder/localdir
Remote Path: GCS:bucket_name_xyz:/remote_folder/
The error message we receive from rclone web ui is as follows:
2023/04/12 04:10:03 ERROR : localdir: Failed to copy: googleapi: Error 400: Use of this bucket name is restricted: 'bucket_name_xyzremote_folder', invalid
It can be seen that, the remote path is problematic. There should be a ":" character between the bucket name and the remote folder name
Download file from bucket to local subfolder: WORKS
Download file from remote subfolder in a bucket to local parent: WORKS
Download file from remote subfolder in a bucket to local subfolder: WORKS
Download folder from bucket to local subfolder FAILS
Local path: local:/subfolder
Remote Path: GCS:bucket_name_xyz:/remote_folder
The error message we receive from rclone web ui is as follows:
2023/04/12 04:15:17 ERROR : GCS bucket bucket_name_xyzremote_folder: error reading source root directory: directory not found
It can be seen that, the remote path is problematic. There should be a ":" character between the bucket name and the remote folder name
Download folder from subfolder in a bucket to local subfolder FAILS
Thanks for your response. I guess, following would be the command line equivalent of what we tried to do on the Web UI. From command line it works as expected:
[user@hostname shared_storage]$ rclone -vvv copy GCS:/bucket_name_xyz/remote_folder/test ./backup/cli_test/
2023/04/12 08:26:52 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "-vvv" "copy" "GCS:/bucket_name_xyz/remote_folder/test" "./backup/cli_test/"]
2023/04/12 08:26:52 DEBUG : Creating backend with remote "GCS:/bucket_name_xyz/remote_folder/test"
2023/04/12 08:26:52 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2023/04/12 08:26:52 DEBUG : fs cache: renaming cache item "GCS:/bucket_name_xyz/remote_folder/test" to be canonical "GCS:bucket_name_xyz/remote_folder/test"
2023/04/12 08:26:52 DEBUG : Creating backend with remote "./backup/cli_test/"
2023/04/12 08:26:52 DEBUG : fs cache: renaming cache item "./backup/cli_test/" to be canonical "/home/user/shared_storage/backup/cli_test"
2023/04/12 08:26:53 DEBUG : Local file system at /home/user/shared_storage/backup/cli_test: Waiting for checks to finish
2023/04/12 08:26:53 DEBUG : Local file system at /home/user/shared_storage/backup/cli_test: Waiting for transfers to finish
2023/04/12 08:26:53 DEBUG : ads: md5 = d41d8cd98f00b204e9800998ecf8427e OK
2023/04/12 08:26:53 INFO : ads: Copied (new)
2023/04/12 08:26:53 DEBUG : asdasd: md5 = d41d8cd98f00b204e9800998ecf8427e OK
2023/04/12 08:26:53 INFO : asdasd: Copied (new)
2023/04/12 08:26:53 DEBUG : new file: md5 = d41d8cd98f00b204e9800998ecf8427e OK
2023/04/12 08:26:53 INFO : new file: Copied (new)
2023/04/12 08:26:53 DEBUG : preAllocate: got error on fallocate, trying combination 1/2: operation not supported
2023/04/12 08:26:53 DEBUG : preAllocate: got error on fallocate, trying combination 2/2: operation not supported
2023/04/12 08:26:53 DEBUG : .upgrade-chrome: md5 = 9a931170ebb7c2adb03c67d6c69a8790 OK
2023/04/12 08:26:53 INFO : .upgrade-chrome: Copied (new)
2023/04/12 08:26:53 INFO :
Transferred: 651 B / 651 B, 100%, 0 B/s, ETA -
Transferred: 4 / 4, 100%
Elapsed time: 0.2s
2023/04/12 08:26:53 DEBUG : 8 go routines active
We are not specifying the paths manually. We are selecting the bucket then the folder of the bucket from the explorer. We will try typing the paths in the explorer's address/path text box and I'll share my results.
Now I see the issue with the extra colon, but I do not get the error - probably because I tested with an S3 backend (instead of GCS).
Seems like the web GUI always adds ":/" to the end of the path if the path doesn't already contain ":/", no matter which remote you are using. This is a bug and can get quite interesting if you try hitting backspace to delete the "/" and then hit enter - I easily managed to make this (local) remote:
myRemote:::::/
The natural next step would therefore be to create a GitHub issue here:
There is just one major problem: The rclone web GUI project seems to be abandoned, the last update was in July 2021:
So I guess you will have to manually work around the problem, or see if you can find a better maintained alternative. I quickly tried but had no luck, sorry.
