Chudz
(Andrew Jones)
June 16, 2023, 12:33pm
1
What is the problem you are having with rclone?
Dropbox/Crypt Mount not polling and refreshing directory cache.
Run the command 'rclone version' and share the full output of the command.
rclone v1.62.2
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.15.0-71-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Dropbox / Crypt
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone mount \
--config=/home/***/.config/rclone/rclone.conf \
--allow-other \
--rc \
--rc-no-auth \
--rc-addr=localhost:5575 \
#--vfs-cache-mode=full \
#--vfs-cache-max-size=200G \
#--vfs-fast-fingerprint \
#--vfs-write-back=1h \
#--vfs-cache-max-age=9999h \
--dir-cache-time=9999h \
--tpslimit=12 \
--tpslimit-burst=0 \
--poll-interval=30s \
--timeout=10m \
--umask=002 \
--log-file=/home/***/logs/dropbox-mount.log \
--log-level=DEBUG \
Dropbox-Crypt: /mnt/dropbox
The rclone config contents with secrets removed.
[Dropbox]
type = dropbox
client_id = ***
client_secret = ***
token = ***
[Dropbox-Crypt]
type = crypt
remote = Dropbox:/
password = ***
password2 = ***
A log from the command with the -vv
flag
2023/06/16 14:35:51 INFO : Starting transaction limiter: max 12 transactions/s with burst 1
2023/06/16 14:35:51 DEBUG : rclone: Version "v1.62.2" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/chudz/.config/rclone/rclone.conf" "--allow-other" "--rc" "--rc-no-auth" "--rc-addr=localhost:5575" "--dir-cache-time=9999h" "--tpslimit=12" "--tpslimit-burst=0" "--poll-interval=30s" "--timeout=10m" "--umask=002" "--log-file=/home/chudz/logs/dropbox-mount.log" "--log-level=DEBUG" "Dropbox-Crypt:" "/mnt/dropbox"]
2023/06/16 14:35:51 NOTICE: Serving remote control on http://127.0.0.1:5575/
2023/06/16 14:35:51 DEBUG : Creating backend with remote "Dropbox-Crypt:"
2023/06/16 14:35:51 DEBUG : Using config file from "/home/chudz/.config/rclone/rclone.conf"
2023/06/16 14:35:51 DEBUG : Creating backend with remote "Dropbox:/"
2023/06/16 14:35:51 DEBUG : Dropbox root '': Using root namespace "4221161601"
2023/06/16 14:35:51 DEBUG : fs cache: renaming cache item "Dropbox:/" to be canonical "Dropbox:"
2023/06/16 14:35:51 DEBUG : Encrypted drive 'Dropbox-Crypt:': Mounting on "/mnt/dropbox"
2023/06/16 14:35:51 DEBUG : : Root:
2023/06/16 14:35:51 DEBUG : : >Root: node=/, err=<nil>
2023/06/16 14:35:51 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true]
2023/06/16 14:35:51 DEBUG : rc: "vfs/refresh": reply map[jobid:1]: <nil>
2023/06/16 14:35:51 DEBUG : : Reading directory tree
2023/06/16 14:35:52 DEBUG : Martina Mendez: Skipping undecryptable dir name: illegal base32 data at input byte 7
2023/06/16 14:36:21 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 14:36:21 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 14:44:35 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 14:44:35 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 14:52:55 INFO : Dropbox root '': Change notify listener failure: upstream request timeout
2023/06/16 14:52:56 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 14:52:56 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 14:55:26 DEBUG : : Reading directory tree done in 19m34.181296951s
2023/06/16 15:01:04 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 15:01:04 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 15:09:12 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 15:09:12 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 15:17:14 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 15:17:14 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
2023/06/16 15:25:33 DEBUG : Dropbox root '': Checking for changes on remote
2023/06/16 15:25:33 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s
asdffdsa
(jojothehumanmonkey)
June 16, 2023, 12:42pm
2
hi, just wanted to be sure you saw this
Martina Mendez: Skipping undecryptable dir name
Chudz
(Andrew Jones)
June 16, 2023, 12:53pm
3
Yeah I seen, not sure if that would cause issues with polling though?
If you mix crypt and non crypt, you will most definitely get issues.
Chudz
(Andrew Jones)
June 16, 2023, 1:01pm
5
That wasn't my intent. The Shared Storage contains only encrypted data.
Martina Mendez is the name of the Dropbox user so that would be their personal directory. I thought adding the Dropbox:/
to the crypt remote would make it only use the Shared Storage?
I will try removing that / and mounting Dropbox-Crypt:/
Chudz
(Andrew Jones)
June 16, 2023, 1:08pm
6
So I switched that around my config is:
[Dropbox]
type = dropbox
client_id = ***
client_secret = ***
token = ***
[Dropbox-Crypt]
type = crypt
remote = Dropbox:
password = ***
password2 = ***
And mount is: Dropbox-Crypt:/ /mnt/dropbox
Still have issue with that personal folder. Maybe I need to create a root directory in the Shared Storage and move everything to that and point the crypt there?
Generally, yes.
You want to point a crypt remote to a specific crypt folder and do not mix things.
Chudz
(Andrew Jones)
June 16, 2023, 4:36pm
8
I moved my encrypted files to an unencrypted directory called Root. It hasn't fixed the issue. Here is my new log:
2023/06/16 19:22:21 INFO : Starting transaction limiter: max 12 transactions/s with burst 1
2023/06/16 19:22:21 DEBUG : rclone: Version "v1.62.2" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/chudz/.config/rclone/rclone.conf" "--allow-other" "--rc" "--rc-no-auth" "--rc-addr=localhost:5575" "--dir-cache-time=9999h" "--tpslimit=12" "--tpslimit-burst=0" "--poll-interval=30s" "--timeout=10m" "--umask=002" "--log-file=/home/chudz/logs/dropbox-mount.log" "--log-level=DEBUG" "Dropbox-Crypt:" "/mnt/dropbox"]
2023/06/16 19:22:21 NOTICE: Serving remote control on http://127.0.0.1:5575/
2023/06/16 19:22:21 DEBUG : Creating backend with remote "Dropbox-Crypt:"
2023/06/16 19:22:21 DEBUG : Using config file from "/home/chudz/.config/rclone/rclone.conf"
2023/06/16 19:22:21 DEBUG : Creating backend with remote "Dropbox:/Root"
2023/06/16 19:22:21 DEBUG : Dropbox root '': Using root namespace "4221161601"
2023/06/16 19:22:22 DEBUG : fs cache: renaming cache item "Dropbox:/Root" to be canonical "Dropbox:Root"
2023/06/16 19:22:22 DEBUG : Encrypted drive 'Dropbox-Crypt:': Mounting on "/mnt/dropbox"
2023/06/16 19:22:22 DEBUG : : Root:
2023/06/16 19:22:22 DEBUG : : >Root: node=/, err=<nil>
2023/06/16 19:22:22 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true]
2023/06/16 19:22:22 DEBUG : rc: "vfs/refresh": reply map[jobid:1]: <nil>
2023/06/16 19:22:22 DEBUG : : Reading directory tree
2023/06/16 19:22:52 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:22:52 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:31:10 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:31:10 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
A bunch of files where uploaded during that 8 minutes. But no changes found.
Here is the rclone.conf:
[Dropbox]
type = dropbox
client_id = ***
client_secret = ***
token = ***
[Dropbox-Crypt]
type = crypt
remote = Dropbox:/Root
password = ***
password2 = ***
Here is the mount:
rclone mount \
--config=/home/chudz/.config/rclone/rclone.conf \
--allow-other \
--rc \
--rc-no-auth \
--rc-addr=localhost:5575 \
#--vfs-cache-mode=full \
#--vfs-cache-max-size=200G \
#--vfs-fast-fingerprint \
#--vfs-write-back=1h \
#--vfs-cache-max-age=9999h \
--dir-cache-time=9999h \
--tpslimit=12 \
--tpslimit-burst=0 \
--poll-interval=30s \
--timeout=10m \
--umask=002 \
--log-file=/home/chudz/logs/dropbox-mount.log \
--log-level=DEBUG \
Dropbox-Crypt: /mnt/dropbox
I actually had a Dropbox account for months and had this issue, I thought the account was bugged or something. I then bought a new one in which the polling worked for a single test file fine. But now moved a bunch of files it stopped working on my new Dropbox account also. It is strange that I have replicated on 2 different Dropbox accounts and no one seems to having the issue.
What's the issue with that log?
Looks like you kicked off a refresh and it'll process and take some time as Dropbox isn't setup for a recursive listing so that first refresh will take time depending on the amount of files and directories you have.
Mine takes ~10 minutes or so the first time.
1 Like
Chudz
(Andrew Jones)
June 16, 2023, 4:45pm
10
Ah I didn't think of that. I will leave it for a few hours to catch up. I have just under 200TB.
Chudz
(Andrew Jones)
June 16, 2023, 5:59pm
11
2023/06/16 19:22:21 INFO : Starting transaction limiter: max 12 transactions/s with burst 1
2023/06/16 19:22:21 DEBUG : rclone: Version "v1.62.2" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/chudz/.config/rclone/rclone.conf" "--allow-other" "--rc" "--rc-no-auth" "--rc-addr=localhost:5575" "--dir-cache-time=9999h" "--tpslimit=12" "--tpslimit-burst=0" "--poll-interval=30s" "--timeout=10m" "--umask=002" "--log-file=/home/chudz/logs/dropbox-mount.log" "--log-level=DEBUG" "Dropbox-Crypt:" "/mnt/dropbox"]
2023/06/16 19:22:21 NOTICE: Serving remote control on http://127.0.0.1:5575/
2023/06/16 19:22:21 DEBUG : Creating backend with remote "Dropbox-Crypt:"
2023/06/16 19:22:21 DEBUG : Using config file from "/home/chudz/.config/rclone/rclone.conf"
2023/06/16 19:22:21 DEBUG : Creating backend with remote "Dropbox:/Root"
2023/06/16 19:22:21 DEBUG : Dropbox root '': Using root namespace "4221161601"
2023/06/16 19:22:22 DEBUG : fs cache: renaming cache item "Dropbox:/Root" to be canonical "Dropbox:Root"
2023/06/16 19:22:22 DEBUG : Encrypted drive 'Dropbox-Crypt:': Mounting on "/mnt/dropbox"
2023/06/16 19:22:22 DEBUG : : Root:
2023/06/16 19:22:22 DEBUG : : >Root: node=/, err=<nil>
2023/06/16 19:22:22 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true]
2023/06/16 19:22:22 DEBUG : rc: "vfs/refresh": reply map[jobid:1]: <nil>
2023/06/16 19:22:22 DEBUG : : Reading directory tree
2023/06/16 19:22:52 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:22:52 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:31:10 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:31:10 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:39:18 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:39:18 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:41:58 DEBUG : : Reading directory tree done in 19m35.677992651s
2023/06/16 19:47:32 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:47:32 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 19:55:35 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 19:55:35 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:03:38 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:03:38 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:11:42 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:11:42 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:19:47 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:19:47 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:28:07 INFO : Dropbox root 'Root': Change notify listener failure: upstream request timeout
2023/06/16 20:28:08 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:28:08 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:36:22 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:36:22 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:44:31 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:44:31 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
2023/06/16 20:52:51 INFO : Dropbox root 'Root': Change notify listener failure: upstream request timeout
2023/06/16 20:52:51 DEBUG : Dropbox root 'Root': Checking for changes on remote
2023/06/16 20:52:51 DEBUG : Dropbox root 'Root': Decreasing poll interval to maximum 480s
I am assuming this is it finishing: 2023/06/16 19:41:58 DEBUG : : Reading directory tree done in 19m35.677992651s
It still not refreshing, stuff is getting uploaded constantly as I'm still filling it.
The issue is: Imgur: The magic of the Internet
That file should of been showing of the mount.
system
(system)
Closed
July 16, 2023, 5:59pm
12
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.