Folders not updating until service is restarted

What is the problem you are having with rclone?

I have a drive mounted with GFS and another drive mounted with Rclone. The problem is that changes done in GFSt mount are not updated in Rclone until the service is restarted. However, I can see in the log that if I create a folder "RCLONETESTII" in GFS it is detected in Rclone log.

On the other hand, the changes done in Rclone are reflected in GFS mount after a few seconds.

What is your rclone version (output from rclone version)

rclone v1.51.0

  • os/arch: windows/amd64
  • go version: go1.13.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10, 64bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command used for mounting the drive

mount --dir-cache-time 72h --log-level DEBUG --log-file=mylogfile.txt --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off gdrive: X: --config "C:\Users\xxx.config\rclone\rclone.conf" --vfs-cache-mode writes

A log from the command with the -vv flag

2020/05/03 13:35:03 DEBUG : rclone: Version "v1.51.0" starting with parameters ["C:\Program Files\rclone\rclone.exe" "mount" "--dir-cache-time" "72h" "--log-level" "DEBUG" "--log-file=mylogfile.txt" "--vfs-read-chunk-size" "128M" "--vfs-read-chunk-size-limit" "off" "gdrive:" "X:" "--config" "C:\Users\xxx\.config\rclone\rclone.conf" "--vfs-cache-mode" "writes"]
2020/05/03 13:35:03 DEBUG : Using config file from "C:\Users\xxx\.config\rclone\rclone.conf"
2020/05/03 13:35:03 DEBUG : Google drive root '': Mounting on "X:"
2020/05/03 13:35:03 DEBUG : vfs cache root is "C:\Windows\system32\config\systemprofile\AppData\Local\rclone\vfs\gdrive"
2020/05/03 13:35:03 DEBUG : Adding path "vfs/forget" to remote control registry
2020/05/03 13:35:03 DEBUG : Adding path "vfs/refresh" to remote control registry
2020/05/03 13:35:03 DEBUG : Adding path "vfs/poll-interval" to remote control registry
2020/05/03 13:35:03 DEBUG : Google drive root '': Mounting with options: ["-o" "fsname=gdrive:" "-o" "subtype=rclone" "-o" "max_readahead=131072" "-o" "attr_timeout=1" "-o" "atomic_o_trunc" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=gdrive"]
2020/05/03 13:35:03 DEBUG : Google drive root '': Init:

2020/05/03 13:35:03 DEBUG : Google drive root '': >Init:
2020/05/03 13:39:03 DEBUG : Google drive root '': Checking for changes on remote
2020/05/03 13:39:03 DEBUG : : changeNotify: relativePath="xxx/Peliculas/TESTRCLONEII", type=2
2020/05/03 13:39:03 DEBUG : : >changeNotify:

Can you share an example of a file not showing up?

Can you rclone ls on the remote and show it's there and ls on the remote?

It normally takes 1 minute by default for things to show up.

Are you doing an upload via rclone to test or how is the file getting there?

I have created a test folder in GFS directly in Windows Explorer.

I have run a Rclone ls in both GFS and Rclone drive, and it's not present in either.

However I can see it there if I run a dir command via cmd or via the GD web explorer:

03/05/2020 13:38 TESTRCLONEII

A test folder won't show up unless there's a file in it. Rclone doesn't sync folders. It syncs files and the folders get created as part of that sync.

I'm sorry but I'm not following as I'm guessing GFS is Google File Stream?

Can you just run through 3 steps?

  • rclone copy file to remote
  • rclone ls remote to see file is there
  • dir on the local machine after 1 minute and see if the file is there

It should look like this:

felix@gemini:/GD$ date
Sun 03 May 2020 09:07:07 AM EDT
felix@gemini:/GD$ rclone copy /etc/hosts gcrypt:
felix@gemini:/GD$ rclone lsl gcrypt:hosts
      266 2020-03-27 08:31:26.338000000 hosts
felix@gemini:/GD$ date
Sun 03 May 2020 09:07:28 AM EDT
felix@gemini:/GD$ ls
mounted  Movies  TV
felix@gemini:/GD$ date
Sun 03 May 2020 09:07:47 AM EDT
felix@gemini:/GD$ ls
hosts  mounted  Movies  TV
felix@gemini:/GD$ ls -al hosts
-rw-rw-r-- 1 felix felix 266 Mar 27 08:31 hosts

My polling is 15 seconds so it shows up faster than 1 minute.

His mount command has "--dir-cache-time 72h". Wouldn't that explain why? My experience (in Windows) is that if I make changes on my read/write mount, they are not reflected on the read-only mount until I prime it again. In my case, this is due to having "--dir-cache-time 1000h" in the read-only mount command. Is that not how it's supposed to work?

For Google Drive, polling should pick up a change. I am not aware that being a Linux only thing.

The dir-cache becomes invalidated with a polling change so it'll recheck.

Hmm, that's not my experience. But what is the point of having dir-cache-time set to such high values if polling invalidates it?

It only invalidates it if a change comes in, otherwise, it keeps it in memory and reduces API hits and speeds things up.

So, in my case, if I only makes changes once a day and then prime the mount immediately afterwards, I could use "--poll-interval 0" to completely disable polling and reduce API hits even more?

And sorry if this is slightly off-topic, but with a read-only mount, could I not also use "--attr-timeout" with a high value to hopefully speed things up a little more? I know you had this in your settings at one point, but removed it fairly quickly.

With 1 billion per day, it really doesn't matter much. I poll every 15 seconds as those are drive.get.changes or some different API hit.

From your scenario, you don't need if you refresh the cache yourself so having the time huge helps and you'd manually refresh it.

I could then get away with this, no?

rclone mount --attr-timeout 1000h --buffer-size 256M --dir-cache-time 1000h --poll-interval 0 --rc --read-only --timeout 1h -v

BTW, I cannnot seem to find --timeout anymore on the rclone site. Does that still exist, and if so, do you think it's still worth to use in my case?

Sorry for turning this into my own support thread, but I'm sure this will help others as well :wink:

EDIT: --timeout does exist

That seems fine for the use case explained.

I use timeout for the pausing and making sure plex works properly if someone pauses for a period of time.

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.