Change of remote mounted on machine A is not being "seen" on same remote on machine B

Hello,

I am using rclone mount to mount my remotes at system start up using a unit in systemd which calls a small script I have written to mount my remotes using the appropriate commands. There is an unmount script also which is called by the systemd unit on stop. Currently the script is mounting remotes on two different services, Koofr and Hidrive. The scripts work great and the remotes are mounted.

I have a "shared" mount on koofr which allows me to mount the remote on different machines in order to share files between different users. As explained the remotes are being mounted successfully on each user machine but,

What is the problem you are having with rclone?

When I make a change to a file or add a new file or folder on machine A it is not seen on machine B unless I issue a refresh folder command on machine B, using pcmanfm
The change is being shown if I log into to remote through the browser and any change is reported there immediately.

Any help/pointers would be gratefully received.

Hilary

What is your rclone version (output from rclone version)

Machine A - Fedora 31
[hwright@localhost ~]$ rclone version
rclone v1.51.0

  • os/arch: linux/amd64
  • go version: go1.13.6
    Machine B - Dedian 10.3
    rclone v1.51.0
  • os/arch: linux/amd64
  • go version: go1.13.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Machine A - Fedora 31, 64
Machine B - Debian 10.3, 64

Which cloud storage system are you using? (eg Google Drive)

Koofr
HiDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount \
				${REMOTE_DEFINITION[1]}: \
				~/${REMOTE_DEFINITION[1]} \
				--daemon \
				--buffer-size 256M \
				--fast-list \
				--drive-chunk-size 128M \
				--log-file ~/.config/rclone/rclone.log \
				--vfs-cache-mode writes

${REMOTE_DEFINITION[1]} = name of the remote = rclone config = local folder
Just to be clear, the remote is mounting correctly on each machine. 


#### A log from the command with the `-vv` flag (eg output from `rclone -vv copy /tmp remote:tmp`)
<!-- Please **DO NOT REDACT** any information except passwords/keys/personal info. -->
Since there is no problem mounting the remotes, I have not included this here.

Do you have logs? And config without secrets.

How long are you waiting?

The default directory caching time is 5 minutes. Koofr doesn't have a change notify system so you'll have to wait for the directory cache to expire to see new files. Or you can issue a vfs/refresh command over the remote control.

Hello,
I tried letting it wait for over 1 hour but to no avail. It just doesn't refresh.
As you suggest, and as I reported, if I do a refresh on machine B it does update the local folder with the contents on the remote.
If I make a change on machine A, the change is visible on the remote immediately, no 5 minute wait. It just doesn't get pushed/pulled to the machine B.
You say that Koofr doesn't have a change notify system. If they did would that solve the issue? Has anyone asked them to provide one? If not I'll write to them, you never know...

If you have logs you could look for the cache expiration entry.

As a new user, the system tells me that I cannot upload files. So here is the log,

2020/04/29 03:30:46 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "mount" "koofr_remote_name:" "/home/myhome/koofr_remote_name" "--daemon" "--buffer-size" "256M" "--fast-list" "--drive-chunk-size" "128M" "--log-file" "/home/myhome/.config/rclone/rclone.log" "--vfs-cache-mode" "writes" "-vv"]
2020/04/29 03:30:46 DEBUG : Using config file from "/home/myhome/.config/rclone/rclone.conf"
2020/04/29 03:30:46 DEBUG : rclone: Version "v1.51.0" finishing with parameters ["rclone" "mount" "koofr_remote_name:" "/home/myhome/koofr_remote_name" "--daemon" "--buffer-size" "256M" "--fast-list" "--drive-chunk-size" "128M" "--log-file" "/home/myhome/.config/rclone/rclone.log" "--vfs-cache-mode" "writes" "-vv"]
2020/04/29 03:30:46 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "mount" "koofr_remote_name:" "/home/myhome/koofr_remote_name" "--daemon" "--buffer-size" "256M" "--fast-list" "--drive-chunk-size" "128M" "--log-file" "/home/myhome/.config/rclone/rclone.log" "--vfs-cache-mode" "writes" "-vv"]
2020/04/29 03:30:46 DEBUG : Using config file from "/home/myhome/.config/rclone/rclone.conf"
2020/04/29 03:30:47 DEBUG : koofr:3xxx571b-xxxx-xxxx-xxxx-063xxxa2f122:: Mounting on "/home/myhome/koofr_remote_name"
2020/04/29 03:30:47 INFO : koofr:3xxx571b-xxxx-xxxx-xxxx-063xxxa2f122:: poll-interval is not supported by this remote
2020/04/29 03:30:47 DEBUG : vfs cache root is "/home/myhome/.cache/rclone/vfs/koofr_remote_name"
2020/04/29 03:30:47 DEBUG : Adding path "vfs/forget" to remote control registry
2020/04/29 03:30:47 DEBUG : Adding path "vfs/refresh" to remote control registry
2020/04/29 03:30:47 DEBUG : Adding path "vfs/poll-interval" to remote control registry
2020/04/29 03:30:47 DEBUG : : Root:
2020/04/29 03:30:47 DEBUG : : >Root: node=/, err=
2020/04/29 03:30:47 DEBUG : /: Attr:
2020/04/29 03:30:47 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:30:48 DEBUG : /: Attr:
2020/04/29 03:30:48 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: Lookup: name="BDMV"
2020/04/29 03:31:14 DEBUG : /: >Lookup: node=, err=no such file or directory
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: Lookup: name=".xdg-volume-info"
2020/04/29 03:31:14 DEBUG : /: >Lookup: node=, err=no such file or directory
2020/04/29 03:31:14 DEBUG : /: Lookup: name="autorun.inf"
2020/04/29 03:31:14 DEBUG : /: >Lookup: node=, err=no such file or directory
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : /: ReadDirAll:
2020/04/29 03:31:14 DEBUG : /: >ReadDirAll: item=9, err=
2020/04/29 03:31:14 DEBUG : /: Lookup: name="autorun.inf"
2020/04/29 03:31:14 DEBUG : /: >Lookup: node=, err=no such file or directory
2020/04/29 03:31:14 DEBUG : /: Attr:
2020/04/29 03:31:14 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2020/04/29 03:31:14 DEBUG : : Statfs:
2020/04/29 03:31:14 DEBUG : : Statfs:
2020/04/29 03:31:14 DEBUG : : >Statfs: stat={Blocks:655360 Bfree:655104 Bavail:655104 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=
2020/04/29 03:31:14 DEBUG : : >Statfs: stat={Blocks:655360 Bfree:655104 Bavail:655104 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=

I upgraded your account. You can upload.

I am beginning to wonder whether what I am expecting to happen is just too optimistic.

Here's the scenario,
I mount the remote on two machines.
I open the remote in the file manager on both machines.
I add a file on one of the machines
I go to the other and would expect to see the file appear automatically without having to do a refresh.

Now, if I do a refresh the file does appear in the folder and it appears almost immediately. I'm experimenting on two machines in the same room.

Do you think that this issue may be more to do with the folder refresh polling (if that exists) of the file manager rather than anything to do with rclone?

rclone.log (9.1 KB)

Thank you for upgrading my account. Here's the log.

File manager. What is displaying is a cached representation of the remote. You'd have to refresh that. If you drop to a shell and ls, is it there?

It should be within 5 minutes.

Yes, it does show with ls in the shell after 5 mins. I had to refresh the file manager to see it.
Is there a way to shorten that delay?

You can expire the cache quicker than 5 minutes. --dir-cache-time. Or, as @ncw said, you can use rc to expire it for the entire mount or a specific directory.

https://rclone.org/commands/rclone_mount/#directory-cache

1 Like

I experimented with adding the --dir-cache-time changing from 2m00s in steps down to 0m01s doing a timing between creating the file on one machine and seeing it with ls on the second machine. Looks like the best I can achieve is using 0m30s or below (I didn't really see any noticeable difference between 30s or 01s) which gives about a 2m delay between creation and availability on second machine. This is acceptable.

Thank you for your time and help with this subject.

I'll close the topic

2 Likes

For info, I wrote the Koofr people and got this reply,

With regard to rclone poll support: we will inspect this and add it to
the rclone Koofr backend in the future if our API can support it in a
reasonable manner.

Let's see...

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.