Leading slash (/) in source/dest paths

This is more a question about the current behaviour so more fitted the Feature section from the description. Apologies if it is a wrong section.

Hi, I just noticed that for

RCLONE_CONFIG_GS_TYPE=google cloud storage
RCLONE_CONFIG_GS_ANONYMOUS=true

Following two URIs seems to work exactly the same with rclone copy

gs:seldon-models/sklearn/iris
gs://seldon-models/sklearn/iris

As I understand properly rclone would in

rclone copy source:sourcepath dest:destpath

treat then gs as source or dest and include the extra leading slashes in the sourcepath or destpath.

Q1. Is rclone then ignoring these or is this remote specific behaviour?
Q2. In context of s3 and and google cloud storage remote types can one rely that gs://seldon-models/sklearn/iris or s3://seldon-models/sklearn/iris will keep working when defined with leading slashes and it is just not side effect of current implementation?

hello,
if you have a question, best to post using the help and support template.
you would have been asked for a debug log that would show rclone fixing the syntax
that is documented here

rclone lsd wasabi01:// -vv 
DEBUG : Using RCLONE_CONFIG_PASS password.
DEBUG : Using config file from "C:\\data\\rclone\\scripts\\rclone.conf"
DEBUG : rclone: Version "v1.55.1" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "lsd" "wasabi01://" "-vv"]
DEBUG : Creating backend with remote "wasabi01://"
DEBUG : fs cache: renaming cache item "wasabi01://" to be canonical "wasabi01:"
1 Like

Thanks, I indeed see similar line

rclone lsd gs://seldon-models -vv
2021/05/04 13:37:47 DEBUG : Using config file from "/home/rskolasinski/.config/rclone/rclone.conf"
2021/05/04 13:37:47 DEBUG : rclone: Version "v1.55.1" starting with parameters ["/home/rskolasinski/.asdf/installs/rclone/1.55.1/bin/rclone" "lsd" "gs://seldon-models" "-vv"]
2021/05/04 13:37:47 DEBUG : Creating backend with remote "gs://seldon-models"
2021/05/04 13:37:47 DEBUG : fs cache: renaming cache item "gs://seldon-models" to be canonical "gs:seldon-models"

so rclone is renaming in the background and thanks to that everything is working as expected.

Can someone consider this behaviour a feature that will be good to rely on in the future? Our URIs are most often defined in gs://... format and thanks to this we do not need to put any compatibility layer in between.

it is already a feature, are you asking for a change in the current behavior?

Apologies if I've been not clear. As I didn't find anything about it in docs (at least not here rclone copy) I am just not sure if this is a "feature" or a "non-documented feature" as latter may in principle change in the future.

So current behaviour is perfect and just hope it will never change :slight_smile:

up above, i posted a link that explains the syntax of remotes.

rclone tries to clean up the syntax if it can.

1 Like

From the link, this is the part that documents the handling of leading slash:

remote:/path/to/dir

On most backends this is refers to the same directory as remote:path/to/dir and that format should be preferred. On a very small number of remotes (FTP, SFTP, Dropbox for business) this will refer to a different directory. On these, paths without a leading / will refer to your "home" directory and paths with a leading / will refer to the root.

But it does not specifically document how two leading slashes (which makes it look like an url) are handled, if thats an important part of the question.

rclone tries to clean up the syntax if it can.

True, and for S3 it does remove all leading and trailing slashes:

And that exact same code seems to be used in most other backend implementations.

So we could perhaps include a short note about this in the documentation, to give @rskolasinski and others peace of mind.. :slight_smile:

1 Like

Would be much appreciated! Thank you guys!