Rclone Mount for window

What is the problem you are having with rclone?

There are 20w objects in the first-level directory of the bucket, and it takes too long to open and load after being mounted to the win local disk.How can I increase the rate of opening files with 200,000 small files (Do not use --no-modtime, --no-checksum)

What is your rclone version (output from rclone version)

Version: v1.55.1 (windows)

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

Paste  log here

hello and welcome to the forum,

this is something that might help but you did not answer the questions from the help template.

since you are new to the forum, i have re-posted the important parts


STOP and READ YOUR POST WILL BE REMOVED IF IT IS LOW QUALITY:
Please show the effort you've put in to solving the problem and please be specific -- people are volunteering their time to help you! Low effort posts are not likely to get good answers! DO NOT REDACT any information except passwords/keys/personal info. You should use 3 backticks to begin and end your paste to make it readable. Or use a service such as https://pastebin.com or https://gist.github.com/

Which cloud storage system are you using? (eg Google Drive)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

The rclone config contents with secrets removed.

Paste config here

rclone version: v1.55.1 (windows)

cloud storage system: S3

command:

rclone.exe mount remote:bucket * --vfs-cache-mode writes --vfs-cache-max-age 1s --dir-cache-time 1s -o FileSecurity=\\"D:P(A;;FA;;;WD)\\" --use-server-modtime --update

log

2021/05/27 09:47:17 DEBUG : /4k_20w: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/27 09:47:17 DEBUG : : Re-reading directory (11.7860557s old)
2021/05/27 09:47:17 DEBUG : /: >Getattr: errc=0
2021/05/27 09:47:17 DEBUG : /: Releasedir: fh=0x5
2021/05/27 09:47:17 DEBUG : /: >Releasedir: errc=0
2021/05/27 09:47:17 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/27 09:47:17 DEBUG : /: >Getattr: errc=0
2021/05/27 09:47:17 DEBUG : /: Opendir:
2021/05/27 09:47:17 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/27 09:47:17 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/27 09:47:17 DEBUG : /: >Getattr: errc=0
2021/05/27 09:47:17 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/27 09:47:17 DEBUG : /: >Getattr: errc=0
2021/05/27 09:47:17 DEBUG : /: Opendir:
2021/05/27 09:47:17 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/27 09:47:17 DEBUG : /4k_20w: >Getattr: errc=0
2021/05/27 09:47:17 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/27 09:47:17 DEBUG : /: >Opendir: errc=0, fh=0x4
2021/05/27 09:47:17 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/27 09:47:17 DEBUG : /4k_20w: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/27 09:47:17 DEBUG : /: Releasedir: fh=0x4
2021/05/27 09:47:17 DEBUG : /: >Opendir: errc=0, fh=0x5
2021/05/27 09:47:17 DEBUG : /4k_20w: >Getattr: errc=0
2021/05/27 09:47:17 DEBUG : /: >Releasedir: errc=0
2021/05/27 09:47:17 DEBUG : /: Releasedir: fh=0x5
2021/05/27 09:47:17 DEBUG : /: >Releasedir: errc=0
2021/05/27 09:47:17 DEBUG : /4k_20w: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/27 09:47:17 DEBUG : /4k_20w: >Getattr: errc=0
2021/05/27 09:47:17 DEBUG : /4k_20w: Opendir:
2021/05/27 09:47:17 DEBUG : /4k_20w: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx

do a pre-scan of the folder structure before you try to access it.

add --rc to the mount command

then after running the mount command but before accessing it, run this command
rclone rc vfs/refresh recursive=true -vv
as documented at https://rclone.org/rc/#vfs-refresh
not sure how much ram will be used.

1 Like

Thank you very much for your answers, but I opened the folder through Windows Explorer after mounting it. Therefore, there is no way to pre-scan after mounting;

ok, let's call it priming the mount, not pre-caching, as @VBB calls it.

  • after the mount is active but before you open the folder in windows explorer, prime the mount.
  • at any time, after the mount is active, you can run the prime command.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.