SMB remote creating a dir named `．`

asdffdsa · November 18, 2022, 9:23pm

What is the problem you are having with rclone?

rclone copy to smb remote create a dir named ．
that is a dot followed by a space character.

could be an issue with encoding
The rclone client tries to avoid 8.3 names when uploading files by encoding trailing spaces and periods

whereas when copying to a rclone mount, does NOT happen.

C:\data\rclone\rclone-v1.60.1-windows-amd64.exe mount nas: b:\rclone\mount\nas

Run the command 'rclone version' and share the full output of the command.

rclone v1.60.1
- os/version: Microsoft Windows 11 Pro 21H2 (64 bit)
- os/kernel: 10.0.22000.1281 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.19.3
- go/linking: static
- go/tags: cmount

Are you on the latest version of rclone?
YES

Which cloud storage system are you using? (eg Google Drive)

SMB

note:
the source is my laptop, W11.2H22.64BIT

the dest is the awesome, free windows server 2019 hyper-v edition
tested on two files systems.
--- NTFS
--- REFS - windows version of ZFS in a raid5 setup.

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone-v1.60.1-windows-amd64.exe copy d:\files\file.ext nas:dest -vv

The rclone config contents with secrets removed.

[nas]
type = smb
host = 192.168.62.233
user = user
pass = pass

A log from the command with the `-vv` flag

rclone-v1.60.1-windows-amd64.exe lsf nas:dest

rclone-v1.60.1-windows-amd64.exe lsd nas:dest

rclone-v1.60.1-windows-amd64.exe tree nas:dest -all
/

0 directories, 0 files

rclone-v1.60.1-windows-amd64.exe copy d:\files\file.ext nas:dest -vv
DEBUG : rclone: Version "v1.60.1" starting with parameters ["C:\\data\\rclone\\rclone-v1.60.1-windows-amd64.exe" "copy" "d:\\files\\file.ext" "nas:dest" "-vv"]
DEBUG : Creating backend with remote "d:\\files\\file.ext"
DEBUG : Using config file from "C:\\data\\rclone\\rclone.conf"
DEBUG : fs cache: adding new entry for parent of "d:\\files\\file.ext", "//?/d:/files"
DEBUG : Creating backend with remote "nas:dest"
DEBUG : file.ext: Need to transfer - File not found at Destination
INFO  : file.ext: Copied (new)
INFO  :
Transferred:              1 B / 1 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.1s

2022/11/18 16:13:05 DEBUG : 8 go routines active
2022/11/18 16:13:05 DEBUG : smb://administrator@192.168.62.233:445/dest/: Closing 2 unused connections

rclone-v1.60.1-windows-amd64.exe lsf nas:dest
file.ext
．/

rclone-v1.60.1-windows-amd64.exe lsd nas:dest
          -1 2022-11-18 16:13:07        -1 ．

rclone-v1.60.1-windows-amd64.exe tree nas:dest -all
/
├── file.ext
└── ．

1 directories, 1 files

and if a picture helps

asdffdsa · November 18, 2022, 9:56pm

same issue:
source: ubuntu on WSL2, same exact remote
dest: same dest as above - windows server

user01@wsl01:~/rclone/scripts$ rclone copy file.ext nas:dest -vv
DEBUG : rclone: Version "v1.60.1" starting with parameters ["rclone" "copy" "file.ext" "nas:dest" "-vv"]
DEBUG : Creating backend with remote "file.ext"
DEBUG : Using config file from "/home/user01/.config/rclone/rclone.conf"
DEBUG : fs cache: adding new entry for parent of "file.ext", "/home/user01/rclone/scripts"
DEBUG : Creating backend with remote "nas:dest"
DEBUG : file.ext: Need to transfer - File not found at Destination
INFO  : file.ext: Copied (new)
INFO  :
Transferred:              2 B / 2 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.2s

DEBUG : 8 go routines active
DEBUG : smb://administrator@192.168.62.233:445/dest/: Closing 2 unused connections

user01@wsl01:~/rclone/scripts$ rclone tree nas:dest
/
├── file.ext
└── ．

asdffdsa · November 18, 2022, 10:23pm

same issue again:
the source is my laptop, W11.2H22.64BIT
the dest is a synology nas over tailscale

[nas_synbox]
type = smb
host = 100.115.37.xxx
user = user
pass = pass

rclone-v1.60.1-windows-amd64.exe lsf nas_synbox:dest

rclone-v1.60.1-windows-amd64.exe lsd nas_synbox:dest

rclone-v1.60.1-windows-amd64.exe tree nas_synbox:dest -all
/

0 directories, 0 files

rclone-v1.60.1-windows-amd64.exe copy d:\files\file.ext nas_synbox:dest -vv
DEBUG : rclone: Version "v1.60.1" starting with parameters ["C:\\data\\rclone\\rclone-v1.60.1-windows-amd64.exe" "copy" "d:\\files\\file.ext" "nas_synbox:dest" "-vv"]
DEBUG : Creating backend with remote "d:\\files\\file.ext"
DEBUG : Using config file from "C:\\data\\rclone\\rclone.conf"
DEBUG : fs cache: adding new entry for parent of "d:\\files\\file.ext", "//?/d:/files"
DEBUG : Creating backend with remote "nas_synbox:dest"
DEBUG : file.ext: Need to transfer - File not found at Destination
INFO  : file.ext: Copied (new)
INFO  :
Transferred:              1 B / 1 B, 100%, 0 B/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:         1.5s

DEBUG : 7 go routines active
DEBUG : smb://user99@100.115.37.34:445/dest/: Closing 2 unused connections

rclone-v1.60.1-windows-amd64.exe lsf nas_synbox:dest
file.ext
．/

rclone-v1.60.1-windows-amd64.exe lsd nas_synbox:dest
          -1 2022-11-18 17:18:52        -1 ．

rclone-v1.60.1-windows-amd64.exe tree nas_synbox:dest -all
/
├── file.ext
└── ．

1 directories, 1 files

ncw · November 18, 2022, 11:05pm

This does look like an encoding issue, that is the special unicode dot rclone uses.

@Lesmiscore do you have any ideas?

asdffdsa · November 18, 2022, 11:08pm

congrats on 10 years!

the strange thing is the issue is not with the single file that was copied.

instead that rclone creates a new dir that is empty.

albertony · November 18, 2022, 11:15pm

I have it in the debugger...

We're getting bitten by the classic golang path.Clean "feature":

If the result of this process is an empty string, Clean returns the string ".".

Therefore, this part:

github.com

rclone/rclone/blob/959cd938bc945c59320a71496d614e71fe4731ae/backend/smb/smb.go#L611


      
          	return in, nil
          }
          
          
// Update the Object from in with modTime and size
          func (o *Object) Update(ctx context.Context, in io.Reader, src fs.ObjectInfo, options ...fs.OpenOption) (err error) {
          	share, filename := o.split()
          	if share == "" || filename == "" {
          		return fs.ErrorIsDir
          	}
          
          
	err = o.fs.ensureDirectory(ctx, share, filename)
          	if err != nil {
          		return fmt.Errorf("failed to make parent directories: %w", err)
          	}
          
          
	filename = o.fs.toSambaPath(filename)
          
          
	o.fs.addSession() // Show session in use
          	defer o.fs.removeSession()
          
          
	cn, err := o.fs.getConnection(ctx, share)

Ends up here:

github.com

rclone/rclone/blob/959cd938bc945c59320a71496d614e71fe4731ae/backend/smb/smb.go#L485


      
          
          
func (f *Fs) makeEntryRelative(share, _path, relative string, stat os.FileInfo) *Object {
          	return f.makeEntry(share, path.Join(_path, relative), stat)
          }
          
          
func (f *Fs) ensureDirectory(ctx context.Context, share, _path string) error {
          	cn, err := f.getConnection(ctx, share)
          	if err != nil {
          		return err
          	}
          	err = cn.smbShare.MkdirAll(f.toSambaPath(path.Dir(_path)), 0o755)
          	f.putConnection(&cn)
          	return err
          }
          
          
/// Object
          
          
// Remote returns the remote path
          func (o *Object) Remote() string {
          	return o.remote
          }

And since path.Dir here performs an implicit Clean, it ends up doing MkdirAll(".") when path is empty.

Edit: Actually, path.Dir returns ".", but then the toSambaPath encodes it, as you indicated, to the unicode dot which is what is supplied to MkdirAll.

albertony · November 19, 2022, 12:04am

Suggested fix:

github.com/rclone/rclone

SMB: Fix issue where spurious dot directory is created

rclone:master ← albertony:smb-fix-dot-dir-issue

opened 11:55PM - 18 Nov 22 UTC

albertony

+5 -1

#### What is the purpose of this change? Given existing code line in the ensu…reDirectory function: ``` cn.smbShare.MkdirAll(f.toSambaPath(path.Dir(_path)), 0o755) ``` When uploading a file to the root of the share, _path here is empty string, then path.Dir will return "." (from its implicit call to path.Clean), toSambaPath will encode it to the Unicode wide equivalent, and the result is as described in the forum thread: A directory `．` is being created, in addition to the uploaded file. Here I check for the "." and aborts before MkdirAll - assuming it is not necessary to call with empty string "". #### Was the change discussed in an issue or in the forum before? https://forum.rclone.org/t/smb-remote-creating-a-dir-named/34188 #### Checklist - [x] I have read the [contribution guidelines](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#submitting-a-new-feature-or-bug-fix). - [ ] I have added tests for all changes in this PR if appropriate. - [ ] I have added documentation for the changes if appropriate. - [x] All commit messages are in [house style](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#commit-messages). - [x] I'm done, this Pull Request is ready for review :-)

ncw · November 20, 2022, 12:25pm

Ah!

We should probably have an rclone.Clean which returns "" instead as that is never what we want!

Anyway I think you fix looks correct - hoping @Lesmiscore can comment.

Lesmiscore · November 20, 2022, 4:02pm

yeah the fix seems correct

ncw · November 20, 2022, 5:12pm

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.61 (and 1.60.2 if we get that far!).

asdffdsa · November 20, 2022, 6:00pm

the beta fixed the issue.

rclone-v1.61.0-beta.6573.7db1c506f-windows-amd64.exe lsf nas:dest

rclone-v1.61.0-beta.6573.7db1c506f-windows-amd64.exe lsd nas:dest

rclone-v1.61.0-beta.6573.7db1c506f-windows-amd64.exe tree nas:dest -all
/

0 directories, 0 files

rclone-v1.61.0-beta.6573.7db1c506f-windows-amd64.exe copy d:\files\file.ext nas:dest -vv
DEBUG : rclone: Version "v1.61.0-beta.6573.7db1c506f" starting with parameters ["C:\\data\\rclone\\rclone-v1.61.0-beta.6573.7db1c506f-windows-amd64.exe" "copy" "d:\\files\\file.ext" "nas:dest" "-vv"]
DEBUG : Creating backend with remote "d:\\files\\file.ext"
DEBUG : Using config file from "C:\\data\\rclone\\rclone.conf"
DEBUG : fs cache: adding new entry for parent of "d:\\files\\file.ext", "//?/d:/files"
DEBUG : Creating backend with remote "nas:dest"
DEBUG : file.ext: Need to transfer - File not found at Destination
INFO  : file.ext: Copied (new)
INFO  :
Transferred:              1 B / 1 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.1s

DEBUG : 8 go routines active
DEBUG : smb://administrator@192.168.62.233:445/dest/: Closing 2 unused connections

rclone-v1.61.0-beta.6573.7db1c506f-windows-amd64.exe lsf nas:dest
file.ext

rclone-v1.61.0-beta.6573.7db1c506f-windows-amd64.exe lsd nas:dest

rclone-v1.61.0-beta.6573.7db1c506f-windows-amd64.exe tree nas:dest -all
/
└── file.ext

0 directories, 1 files

ncw · November 20, 2022, 6:01pm

Great Thanks for testing and thanks to @albertony for fixing.

system · November 23, 2022, 6:01pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.