Issue with Single Character Configuration on Windows with rclone

Umiiii · August 31, 2023, 3:03am

What is the problem you are having with rclone?

When utilizing rclone with a configuration named "o", the software incorrectly renames the configuration path from "o:/post/" to "//?/o:/post". Consequently, this results in failure in subsequent steps due to the system being unable to locate the specified directory.

Run the command 'rclone version' and share the full output of the command.

v1.63.1

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

Steps to Reproduce:

Run the following command:

powershellCopy code

.\copy.ps1

This script invokes rclone with the following parameters:

rclone sync -P o:/post/ d:/gcloud --transfers=20 -vv

The rclone config contents with secrets removed.

[o]
type = drive
client_id = .apps.googleusercontent.com
client_secret = GOCSPX-
scope = drive
root_folder_id =
token = {"access_token":"","token_type":"Bearer","refresh_token":"","expiry":"2023-08-31T00:42:48.878933953+08:00"}

A log from the command with the `-vv` flag

PS D:\> .\copy.ps1
2023/08/30 23:53:54 DEBUG : rclone: Version "v1.63.1" starting with parameters ["D:\\rclone.exe" "sync" "-P" "o:/post/" "d:/gcloud" "--transfers=20" "-vv"]
2023/08/30 23:53:54 DEBUG : Creating backend with remote "o:/post/"
2023/08/30 23:53:54 DEBUG : Using config file from "D:\\rclone.conf"
2023/08/30 23:53:54 DEBUG : fs cache: renaming cache item "o:/post/" to be canonical "//?/o:/post"
2023/08/30 23:53:54 DEBUG : Creating backend with remote "d:/gcloud"
2023/08/30 23:53:54 DEBUG : fs cache: renaming cache item "d:/gcloud" to be canonical "//?/d:/gcloud"
2023-08-30 23:53:54 ERROR : Local file system at //?/o:/post: error reading source root directory: directory not found
2023-08-30 23:53:54 DEBUG : Local file system at //?/d:/gcloud: Waiting for checks to finish
2023-08-30 23:53:54 DEBUG : Local file system at //?/d:/gcloud: Waiting for transfers to finish
2023-08-30 23:53:54 ERROR : Local file system at //?/d:/gcloud: not deleting files as there were IO errors
2023-08-30 23:53:54 ERROR : Local file system at //?/d:/gcloud: not deleting directories as there were IO errors
2023-08-30 23:53:54 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2023-08-30 23:53:54 ERROR : Local file system at //?/o:/post: error reading source root directory: directory not found
2023-08-30 23:53:54 DEBUG : Local file system at //?/d:/gcloud: Waiting for checks to finish
2023-08-30 23:53:54 DEBUG : Local file system at //?/d:/gcloud: Waiting for transfers to finish
2023-08-30 23:53:54 ERROR : Local file system at //?/d:/gcloud: not deleting files as there were IO errors
2023-08-30 23:53:54 ERROR : Local file system at //?/d:/gcloud: not deleting directories as there were IO errors
2023-08-30 23:53:54 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2023-08-30 23:53:54 ERROR : Local file system at //?/o:/post: error reading source root directory: directory not found
2023-08-30 23:53:54 DEBUG : Local file system at //?/d:/gcloud: Waiting for checks to finish
2023-08-30 23:53:54 DEBUG : Local file system at //?/d:/gcloud: Waiting for transfers to finish
2023-08-30 23:53:54 ERROR : Local file system at //?/d:/gcloud: not deleting files as there were IO errors
2023-08-30 23:53:54 ERROR : Local file system at //?/d:/gcloud: not deleting directories as there were IO errors
2023-08-30 23:53:54 ERROR : Attempt 3/3 failed with 1 errors and: directory not found

Plus:
When I renamed it to "oo", it works.

kapitainsky · August 31, 2023, 8:11am

You are right but I am not sure if this should be considered a bug.

Problem is that in Windows there is ambiguity what to do when rclone has to access e.g. c:/path and both c remote and c drive exist. It look that rclone choses to treat it as a drive letter.

IMO it might be worth to put short note in documentation to clarify that in Windows user should not use one letter remotes' names. PR welcomed.

Umiiii · August 31, 2023, 8:31am

How about like this:

github.com

rclone/rclone/blob/11eeaaf792776b598f80e1416f5c9629840e16d8/fs/cache/cache.go#L79


      
          		created = ok
          		return f, ok, err
          	})
          	if err != nil && err != fs.ErrorIsFile {
          		return nil, err
          	}
          	f = value.(fs.Fs)
          	// Check we stored the Fs at the canonical name
          	if created {
          		canonicalName := fs.ConfigString(f)
          		if canonicalName != canonicalFsString {
          			// Note that if err == fs.ErrorIsFile at this moment
          			// then we can't rename the remote as it will have the
          			// wrong error status, we need to add a new one.
          			if err == nil {
          				fs.Debugf(nil, "fs cache: renaming cache item %q to be canonical %q", canonicalFsString, canonicalName)
          				value, found := c.Rename(canonicalFsString, canonicalName)
          				if found {
          					f = value.(fs.Fs)
          				}
          				addMapping(canonicalFsString, canonicalName)

pseudocode

   .... original code ....
If sys == windows:
   if configuration name is single letter && this letter doesn't have an existing drive assigned:
        do not rename it
   .... original code ....

kapitainsky · August 31, 2023, 8:39am

It might be an option too.
Unfortunately it also means that rclone behaviour will differ depending on e.g. if external disk is connected or not. And IMO it should not.

Me personally I would avoid using single letter remotes' names in Windows and document it.

But let's see what others think.

ncw · August 31, 2023, 9:06am

I'm sure there is a note in some doc somewhere about this!

It should probably go here

Umiiii:

pseudocode

   .... original code ....
If sys == windows:
   if configuration name is single letter && this letter doesn't have an existing drive assigned:
        do not rename it
   .... original code ....

I don't want to do that as it will mean we can't parse remote names without some very OS specific code for Windows.

It would be easier to do the reverse and see whether we have a remote called o before parsing o:\path as a disk.

We could do an error/warning on windows if you use rclone config to create a single letter remote?

kapitainsky · August 31, 2023, 9:15am

and if I name remote C it will make accessing disk C impossible - it will become too specific with many edge cases. Much cleaner is simply do not use one letter remotes' names in Windows - it is this platform limitation.

It is also only half solution. Config can be copy/paste from other platform configuration.

IMO we should leave it as it is - one letter name on Windows is treated as a drive. and document it.

ncw · August 31, 2023, 10:21am

That works for me

kapitainsky · August 31, 2023, 10:23am

@Umiiii - what you think?

Umiiii · August 31, 2023, 1:10pm

ok, agree with you.

kapitainsky · August 31, 2023, 1:24pm

asdffdsa · August 31, 2023, 1:42pm

IMO, rclone should never allow single character remote and fatal error.
there have been a number forum posts from newbies making mistakes.

c: on windows is very bad form, dangerous and is never required.
no tool targeting business should allow it.

fwiw, dumb dos will fatal error...

c:\singlechartest>copy c: nul
c:
The system cannot find the file specified.
        0 file(s) copied.

c:\singlechartest>copy . nul
.\file.ext
        1 file(s) copied.

ncw · August 31, 2023, 3:42pm

We could ban single letter remotes or warn about them. Exactly where though and on which platforms

in rclone config when creating new remotes. This is backwards compatible.
- this could error on Windows and warn on non-Windows?
whenever rclone parses a remote name. This will kill all of them. Not backwards compatible!

asdffdsa · August 31, 2023, 4:10pm

in the end, only so much you can do or should have do to.
once we update the docs, the onus is on the user.

yes and might apply to rclone config create

for advanced features for advanced users, such on-the-fly remotes, as when using environment variables and/or connections strings,
user should know to use --dry-run and read the output; so a warning should be enough.

ncw · September 1, 2023, 10:16am

Actually we have this on Windows already

C:\Users\Dev>rclone config
...
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n
name> s
Can't use "s" as it can be confused with a drive letter.
name>

Given that this hasn't come up before in the last 10 years, I'm going to say that plus @kapitainsky 's doc fix is enough!

Umiiii · September 1, 2023, 12:47pm

but sometimes people may copied their configuration from non-windows rclone (like me)

system · September 4, 2023, 12:48pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.