Rclone mount (with crypt) randomly loses directory entries

Help and Support
1

What is the problem you are having with rclone?

I'm using rclone mount with the vfs cache, and it randomly loses directory entries. I know the files are there (can see them in gdrive ui or with an rclone ls) and when I unmount (and stop all process using the mount, so not best solution) and remount they are there. I tried various rclone rc vfs commands, but nothing seemed ot make a difference besides unmount / mount

What is your rclone version (output from rclone version)

$ rclone version
rclone v1.53.3

  • os/arch: linux/amd64
  • go version: go1.15.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

linux (ubunty 20.10)

Which cloud storage system are you using? (eg Google Drive)

google drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

mount i use

rclone mount --cache-dir /home/spotter/rclone-cache --allow-other --no-modtime --attr-timeout 8700h --stats 10s --buffer-size 16M --vfs-cache-poll-interval 30s --poll-interval 30s --dir-cache-time 8700h --vfs-cache-max-age 96h --async-read=true --vfs-read-wait 30ms --vfs-write-wait 30s --vfs-cache-mode full --vfs-read-chunk-size 16M --vfs-read-chunk-size-limit 1G --local-no-check-updated --drive-chunk-size=64M --multi-thread-streams=4 --multi-thread-cutoff 250M --transfers 64 --drive-disable-http2=true --fast-list --vfs-case-insensitive --vfs-cache-max-size 0.9T --log-level ERROR --log-file=/tmp/rc-mount.log --rc gcrypt: /data/gdrive/

The rclone config contents with secrets removed.

type = drive
client_id = 
client_secret = 
token =
root_folder_id =

i.e. nothing special in config.

A log from the command with the -vv flag

don't have a useful config as in this case, wasn't running with any real logging and it happens randomly. sometimes I dont see it for weeks.

2

Are they filed that have been uploaded? Files that have been there before and just are suddenly gone?

3

these were files that were rclone moved to a temp dir on the remote, I'm pretty sure I saw them in the temp dir via the mount (but might be mistaken), but were then moved to their permanent location and then the files were missing in the mount (but as I said, visible in the ui and via rclone ls)

4

You may want to try the beta or wait for 1.54 as there was a fix for items in the cache going missing:

It's closed, but in the latest beta and set to release in 1.54.

When you say moved, did you mean moved from the cache or a server side move? I'm not aware of any polling items for GD still open.

5

per the above mount desriptions

I do rclone move to gcrypt:/ToSort
I then go /data/gdrive/ToSort and simple "mv /data/gdrive/"

6

ok, upgraded to most recent beta will see if it reoccurs

7

so a related thing has happened. waiting to see if it recovers

$ ls Season\ 1/*
'Season 1/a:
a.mkv

'Season 1/b':
b.mkv

'Season 1/c':
c.mkv

'Season 1/d':
d.mkv

'Season 1/e':
e.mkv

'Season 1/f':
f.mkv

$ mv Season\ 1/ new-name
$ cd new-name
$ ls
a
b
c
d
e
f
$ mkdir 1
$ ls
1
a
b
c
d
e
f

$ mv */*mkv 1/

mv: cannot stat '*/*mkv': No such file or directory
$ ls *
1:

a:

b:

c:

d:

e:

f:
8

after an incredibly long delay, it finally refreshed the dir and I can see the contents.

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.