Dropbox/Crypt Mount not polling and refreshing directory cache

Help and Support
1

What is the problem you are having with rclone?

Dropbox/Crypt Mount not polling and refreshing directory cache.

Run the command 'rclone version' and share the full output of the command.

rclone v1.62.2
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.15.0-71-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Dropbox / Crypt

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount \
  --config=/home/***/.config/rclone/rclone.conf \
  --allow-other \
  --rc \
  --rc-no-auth \
  --rc-addr=localhost:5575 \
  #--vfs-cache-mode=full \
  #--vfs-cache-max-size=200G \
  #--vfs-fast-fingerprint \
  #--vfs-write-back=1h \
  #--vfs-cache-max-age=9999h \
  --dir-cache-time=9999h \
  --tpslimit=12 \
  --tpslimit-burst=0 \
  --poll-interval=30s \
  --timeout=10m \
  --umask=002 \
  --log-file=/home/***/logs/dropbox-mount.log \
  --log-level=DEBUG \
  Dropbox-Crypt: /mnt/dropbox

The rclone config contents with secrets removed.

[Dropbox]
type = dropbox
client_id = ***
client_secret = ***
token = ***

[Dropbox-Crypt]
type = crypt
remote = Dropbox:/
password = ***
password2 = ***

A log from the command with the -vv flag

2023/06/16 14:35:51 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/06/16 14:35:51 DEBUG : rclone: Version "v1.62.2" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/chudz/.config/rclone/rclone.conf" "--allow-other" "--rc" "--rc-no-auth" "--rc-addr=localhost:5575" "--dir-cache-time=9999h" "--tpslimit=12" "--tpslimit-burst=0" "--poll-interval=30s" "--timeout=10m" "--umask=002" "--log-file=/home/chudz/logs/dropbox-mount.log" "--log-level=DEBUG" "Dropbox-Crypt:" "/mnt/dropbox"]
2023/06/16 14:35:51 NOTICE: Serving remote control on http://127.0.0.1:5575/
2023/06/16 14:35:51 DEBUG : Creating backend with remote "Dropbox-Crypt:"
2023/06/16 14:35:51 DEBUG : Using config file from "/home/chudz/.config/rclone/rclone.conf"
2023/06/16 14:35:51 DEBUG : Creating backend with remote "Dropbox:/"
2023/06/16 14:35:51 DEBUG : Dropbox root '': Using root namespace "4221161601"
2023/06/16 14:35:51 DEBUG : fs cache: renaming cache item "Dropbox:/" to be canonical "Dropbox:"
2023/06/16 14:35:51 DEBUG : Encrypted drive 'Dropbox-Crypt:': Mounting on "/mnt/dropbox"
2023/06/16 14:35:51 DEBUG : : Root:
2023/06/16 14:35:51 DEBUG : : >Root: node=/, err=<nil>
2023/06/16 14:35:51 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true]
2023/06/16 14:35:51 DEBUG : rc: "vfs/refresh": reply map[jobid:1]: <nil>
2023/06/16 14:35:51 DEBUG : : Reading directory tree
2023/06/16 14:35:52 DEBUG : Martina Mendez: Skipping undecryptable dir name: illegal base32 data at input byte 7
2023/06/16 14:36:21 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 14:36:21 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 14:44:35 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 14:44:35 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 14:52:55 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2023/06/16 14:52:56 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 14:52:56 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 14:55:26 DEBUG : : Reading directory tree done in 19m34.181296951s
2023/06/16 15:01:04 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 15:01:04 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 15:09:12 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 15:09:12 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 15:17:14 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 15:17:14 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 15:25:33 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 15:25:33 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2

hi, just wanted to be sure you saw this
Martina Mendez: Skipping undecryptable dir name

3

Yeah I seen, not sure if that would cause issues with polling though?

4

If you mix crypt and non crypt, you will most definitely get issues.

5

That wasn't my intent. The Shared Storage contains only encrypted data.

Martina Mendez is the name of the Dropbox user so that would be their personal directory. I thought adding the Dropbox:/ to the crypt remote would make it only use the Shared Storage?

I will try removing that / and mounting Dropbox-Crypt:/

6

So I switched that around my config is:

[Dropbox]
type = dropbox
client_id = ***
client_secret = ***
token = ***

[Dropbox-Crypt]
type = crypt
remote = Dropbox:
password = ***
password2 = ***

And mount is: Dropbox-Crypt:/ /mnt/dropbox

Still have issue with that personal folder. Maybe I need to create a root directory in the Shared Storage and move everything to that and point the crypt there?

7

Generally, yes.

You want to point a crypt remote to a specific crypt folder and do not mix things.

8

I moved my encrypted files to an unencrypted directory called Root. It hasn't fixed the issue. Here is my new log:

2023/06/16 19:22:21 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/06/16 19:22:21 DEBUG : rclone: Version "v1.62.2" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/chudz/.config/rclone/rclone.conf" "--allow-other" "--rc" "--rc-no-auth" "--rc-addr=localhost:5575" "--dir-cache-time=9999h" "--tpslimit=12" "--tpslimit-burst=0" "--poll-interval=30s" "--timeout=10m" "--umask=002" "--log-file=/home/chudz/logs/dropbox-mount.log" "--log-level=DEBUG" "Dropbox-Crypt:" "/mnt/dropbox"]
2023/06/16 19:22:21 NOTICE: Serving remote control on http://127.0.0.1:5575/
2023/06/16 19:22:21 DEBUG : Creating backend with remote "Dropbox-Crypt:"
2023/06/16 19:22:21 DEBUG : Using config file from "/home/chudz/.config/rclone/rclone.conf"
2023/06/16 19:22:21 DEBUG : Creating backend with remote "Dropbox:/Root"
2023/06/16 19:22:21 DEBUG : Dropbox root '': Using root namespace "4221161601"
2023/06/16 19:22:22 DEBUG : fs cache: renaming cache item "Dropbox:/Root" to be canonical "Dropbox:Root"
2023/06/16 19:22:22 DEBUG : Encrypted drive 'Dropbox-Crypt:': Mounting on "/mnt/dropbox"
2023/06/16 19:22:22 DEBUG : : Root:
2023/06/16 19:22:22 DEBUG : : >Root: node=/, err=<nil>
2023/06/16 19:22:22 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true]
2023/06/16 19:22:22 DEBUG : rc: "vfs/refresh": reply map[jobid:1]: <nil>
2023/06/16 19:22:22 DEBUG : : Reading directory tree
2023/06/16 19:22:52 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:22:52 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:31:10 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:31:10 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s

A bunch of files where uploaded during that 8 minutes. But no changes found.

Here is the rclone.conf:

[Dropbox]
type = dropbox
client_id = ***
client_secret = ***
token = ***

[Dropbox-Crypt]
type = crypt
remote = Dropbox:/Root
password = ***
password2 = ***

Here is the mount:

rclone mount \
  --config=/home/chudz/.config/rclone/rclone.conf \
  --allow-other \
  --rc \
  --rc-no-auth \
  --rc-addr=localhost:5575 \
  #--vfs-cache-mode=full \
  #--vfs-cache-max-size=200G \
  #--vfs-fast-fingerprint \
  #--vfs-write-back=1h \
  #--vfs-cache-max-age=9999h \
  --dir-cache-time=9999h \
  --tpslimit=12 \
  --tpslimit-burst=0 \
  --poll-interval=30s \
  --timeout=10m \
  --umask=002 \
  --log-file=/home/chudz/logs/dropbox-mount.log \
  --log-level=DEBUG \
  Dropbox-Crypt: /mnt/dropbox

I actually had a Dropbox account for months and had this issue, I thought the account was bugged or something. I then bought a new one in which the polling worked for a single test file fine. But now moved a bunch of files it stopped working on my new Dropbox account also. It is strange that I have replicated on 2 different Dropbox accounts and no one seems to having the issue.

9

What's the issue with that log?

Looks like you kicked off a refresh and it'll process and take some time as Dropbox isn't setup for a recursive listing so that first refresh will take time depending on the amount of files and directories you have.

Mine takes ~10 minutes or so the first time.

1 Like
10

Ah I didn't think of that. I will leave it for a few hours to catch up. I have just under 200TB.

11 
2023/06/16 19:22:21 INFO  : Starting transaction limiter: max 12 transactions/s with burst 1
2023/06/16 19:22:21 DEBUG : rclone: Version "v1.62.2" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/chudz/.config/rclone/rclone.conf" "--allow-other" "--rc" "--rc-no-auth" "--rc-addr=localhost:5575" "--dir-cache-time=9999h" "--tpslimit=12" "--tpslimit-burst=0" "--poll-interval=30s" "--timeout=10m" "--umask=002" "--log-file=/home/chudz/logs/dropbox-mount.log" "--log-level=DEBUG" "Dropbox-Crypt:" "/mnt/dropbox"]
2023/06/16 19:22:21 NOTICE: Serving remote control on http://127.0.0.1:5575/
2023/06/16 19:22:21 DEBUG : Creating backend with remote "Dropbox-Crypt:"
2023/06/16 19:22:21 DEBUG : Using config file from "/home/chudz/.config/rclone/rclone.conf"
2023/06/16 19:22:21 DEBUG : Creating backend with remote "Dropbox:/Root"
2023/06/16 19:22:21 DEBUG : Dropbox root '': Using root namespace "4221161601"
2023/06/16 19:22:22 DEBUG : fs cache: renaming cache item "Dropbox:/Root" to be canonical "Dropbox:Root"
2023/06/16 19:22:22 DEBUG : Encrypted drive 'Dropbox-Crypt:': Mounting on "/mnt/dropbox"
2023/06/16 19:22:22 DEBUG : : Root:
2023/06/16 19:22:22 DEBUG : : >Root: node=/, err=<nil>
2023/06/16 19:22:22 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true]
2023/06/16 19:22:22 DEBUG : rc: "vfs/refresh": reply map[jobid:1]: <nil>
2023/06/16 19:22:22 DEBUG : : Reading directory tree
2023/06/16 19:22:52 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:22:52 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:31:10 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:31:10 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:39:18 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:39:18 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:41:58 DEBUG : : Reading directory tree done in 19m35.677992651s
2023/06/16 19:47:32 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:47:32 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:55:35 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:55:35 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:03:38 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:03:38 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:11:42 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:11:42 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:19:47 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:19:47 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:28:07 INFO  : Dropbox root 'Root': Change notify listener failure: upstream request timeout
2023/06/16 20:28:08 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:28:08 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:36:22 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:36:22 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:44:31 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:44:31 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:52:51 INFO  : Dropbox root 'Root': Change notify listener failure: upstream request timeout
2023/06/16 20:52:51 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:52:51 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s

I am assuming this is it finishing: 2023/06/16 19:41:58 DEBUG : : Reading directory tree done in 19m35.677992651s

It still not refreshing, stuff is getting uploaded constantly as I'm still filling it.

The issue is: Imgur: The magic of the Internet
That file should of been showing of the mount.

12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.