Following two URIs seems to work exactly the same with rclone copy
gs:seldon-models/sklearn/iris
gs://seldon-models/sklearn/iris
As I understand properly rclone would in
rclone copy source:sourcepath dest:destpath
treat then gs as source or dest and include the extra leading slashes in the sourcepath or destpath.
Q1. Is rclone then ignoring these or is this remote specific behaviour?
Q2. In context of s3 and and google cloud storage remote types can one rely that gs://seldon-models/sklearn/iris or s3://seldon-models/sklearn/iris will keep working when defined with leading slashes and it is just not side effect of current implementation?
hello,
if you have a question, best to post using the help and support template.
you would have been asked for a debug log that would show rclone fixing the syntax
that is documented here
rclone lsd wasabi01:// -vv
DEBUG : Using RCLONE_CONFIG_PASS password.
DEBUG : Using config file from "C:\\data\\rclone\\scripts\\rclone.conf"
DEBUG : rclone: Version "v1.55.1" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "lsd" "wasabi01://" "-vv"]
DEBUG : Creating backend with remote "wasabi01://"
DEBUG : fs cache: renaming cache item "wasabi01://" to be canonical "wasabi01:"
rclone lsd gs://seldon-models -vv
2021/05/04 13:37:47 DEBUG : Using config file from "/home/rskolasinski/.config/rclone/rclone.conf"
2021/05/04 13:37:47 DEBUG : rclone: Version "v1.55.1" starting with parameters ["/home/rskolasinski/.asdf/installs/rclone/1.55.1/bin/rclone" "lsd" "gs://seldon-models" "-vv"]
2021/05/04 13:37:47 DEBUG : Creating backend with remote "gs://seldon-models"
2021/05/04 13:37:47 DEBUG : fs cache: renaming cache item "gs://seldon-models" to be canonical "gs:seldon-models"
so rclone is renaming in the background and thanks to that everything is working as expected.
Can someone consider this behaviour a feature that will be good to rely on in the future? Our URIs are most often defined in gs://... format and thanks to this we do not need to put any compatibility layer in between.
Apologies if I've been not clear. As I didn't find anything about it in docs (at least not here rclone copy) I am just not sure if this is a "feature" or a "non-documented feature" as latter may in principle change in the future.
From the link, this is the part that documents the handling of leading slash:
remote:/path/to/dir
On most backends this is refers to the same directory as remote:path/to/dir and that format should be preferred. On a very small number of remotes (FTP, SFTP, Dropbox for business) this will refer to a different directory. On these, paths without a leading / will refer to your "home" directory and paths with a leading / will refer to the root.
But it does not specifically document how two leading slashes (which makes it look like an url) are handled, if thats an important part of the question.
rclone tries to clean up the syntax if it can.
True, and for S3 it does remove all leading and trailing slashes:
And that exact same code seems to be used in most other backend implementations.
So we could perhaps include a short note about this in the documentation, to give @rskolasinski and others peace of mind..
After some consideration it was decided not to do this.
The conclusion is: Leading (and trailing) slashes are currently ignored (removed) for most backends. But you cannot rely on this as a general feature, and it may change in future versions. S3, and some other providers, actually support slash as part of the name, which means if rclone were to support this then references to //path and /path could give different results.
