Deletions from mount not working across two systems when using VFS Full

What is the problem you are having with rclone?

I've previously reported an issue with server-side moves where changes are not being picked up when two servers are using an rclone mount to the same Gdrive account here - Polling/mount cache issue after server-side move

However, I think I have found a new bug since switching to 1.53.1 and full VFS mode.

I have two systems which have a mount to a common GDrive account - one is a seedbox where I run Sonarr/Radarr etc which controls what's on the Drive mount, ie I do all the downloads/uploads/deletes here. I then have a VPS which runs Plex, with a mount to the same Gdrive account.

Since 1.53.1/VFS=Full, I have seen a few instances where Sonarr downloads an upgraded file for a particular show and deletes the original downloaded file. All works ok on the Seedbox, but on the VPS mount, I can see the new file but the old one is still there, ie it hasn't picked up the delete done by the seedbox. Plex then signals there are two files for the episode. I can manually delete the file that should have been deleted from the mount on the VPS and that puts it back into sync.

Doing some testing, I have reproduced the issue and the circumstances that cause it. If I create a file on the seedbox, the VPS picks it up. Next, and this is the key part, the file is opened/played on the VPS so it forces rclone to put it into the local VFS cache. If I then go back to the seedbox and delete the file, it does NOT get deleted on the VPS. It looks like once it's in the VFS cache, deletes on another system do not cause it to be removed from the VFS cache on the VPS. I proved this by repeating the test but NOT opening the file on the VPS so rclone doesn't put it into the VFS cache - if I then delete it on the seedbox, it gets deleted on the VPS also.

Full logs are attached and it seems the VPS does pick up the delete by polling, but I'm guessing rclone does NOT then remove it from the VFS cache, and so on the mount, it still appears to exist. Interestingly, I also have --dir-cache-time 24h and was expecting that if I left it for 24 hours, the VPS would come into sync, but this does not happen - it still shows the file as existing after 24 hrs. I also am not specifying --vfs-cache-max-age so that's defaulting to an hour and that also is not causing the file to be removed from the cache. However, stopping/starting rclone on the VPS does then no longer show the deleted file.

Hope all this makes sense, but it does look like a bug.

What is your rclone version (output from rclone version)

rclone v1.53.1

  • os/arch: linux/amd64
  • go version: go1.15

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 18.04.5 LTS 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount Gsuite-Crypt: /home/lauried/files/Test-mount/ --dir-cache-time 24h --allow-other --drive-chunk-size 512M --vfs-cache-mode full --log-file /home/lauried/files/rclone-test-mount.log --user-agent Laurie --buffer-size 256M --config /home/lauried/files/rclone-test.conf -vv --poll-interval 30s

The rclone config contents with secrets removed.

[Gsuite]
type = drive
client_id = xxxxxx
client_secret = xxxxxx
scope = drive
root_folder_id = xxxxxx
service_account_file = 
token = {"access_token":"xxxxxx"}

[Gsuite-Crypt]
type = crypt
remote = Gsuite:Encrypted
filename_encryption = standard
password = xxxxxxx
password2 = xxxxxxx


A log from the command with the -vv flag

https://wetransfer.com/downloads/12025002a9c9cd1f32d24904269a0d4520201007105555/6ade460010c1d95ec56777157b9aa82820201007105614/9ed286

This is also a known issue:

Doh, sorry, I searched to see if this had been reported and for some reason didn't see this :face_with_raised_eyebrow:

Glad that @ncw is trying to fix it, I'll just wait for now,,,,

Yes I think all the issues have the same basic cause - the virtual directory entries, which clearly I need to rethink.

Low on inspiration at the moment though as personal life rather challenging :frowning:

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.