Dropbox rclone mount Error messages?

What is the problem you are having with rclone?

I encountered some error messages and I dont know how to interpret them correctly.
I got an EOF Error, but as far as I know there was no Problem when accessing the File.
And i dont know what all these upstream timeouts mean and what can be done to fix them

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.0

  • os/version: ubuntu 22.04 (64 bit)
  • os/kernel: 5.15.0-43-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.18.3
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Dropbox

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/usr/bin/rclone mount \
  --user-agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36' \
  --config=/home/sbuser/.config/rclone/rclone.conf \
  --allow-other \
  --allow-non-empty \
  --rc \
  --rc-no-auth \
  --rc-addr=localhost:5572 \
  --drive-skip-gdocs \
  --vfs-read-chunk-size=64M \
  --vfs-read-chunk-size-limit=2048M \
  --buffer-size=64M \
  --poll-interval=15s \
  --dir-cache-time=1000h \
  --timeout=10m \
  --drive-chunk-size=64M \
  --drive-pacer-min-sleep=10ms \
  --drive-pacer-burst=1000 \
##### Dropbox stuff
  --dropbox-chunk-size=64M \
  --tpslimit=10 \
  --tpslimit-burst=10 \
  --disable-http2 \
######
  --umask=002 \
  --log-file=/opt/rclone/rclone_vfs.log \
#  --syslog \
  -v \
  google: /mnt/remote

The rclone config contents with secrets removed.

[google-Movies]
type = drive
service_account_file = /opt/sa/all/100.json
team_drive = XXX
scope = drive

[google-Music]
type = drive
scope = drive
service_account_file = /opt/sa/all/150.json
team_drive = XXX

[google-TV]
type = drive
team_drive = XXX
scope = drive
service_account_file = /opt/sa/all/200.json

[movies-crypt4]
type = crypt
remote = google-Movies:
password = XXX
password2 = YYY

[music-crypt]
type = crypt
remote = google-Music:
password = XXX
password2 =  YYY

[tv-crypt4]
type = crypt
remote = google-TV:
password = XXX	
password2 = YYY

[google]
type = union
upstreams = movies-crypt4:/ music-crypt:/ db-movies-crypt: db-tv-crypt:


[db-movies]
type = dropbox
client_id = XXX
client_secret = YYY
token = {"access_token":"TOKEN","token_type":"bearer","refresh_token":"TOKEN","expiry":"2022-08-11T11:58:49.806265142+02:00"}

[db-tv]
type = dropbox
client_id = XXX
client_secret = YYY
token = {"access_token":"TOKEN","token_type":"bearer","refresh_token":"TOKEN","expiry":"2022-08-11T13:58:20.018587445+02:00"}

[db-movies-crypt]
type = crypt
remote = db-movies:
password = XXX
password2 = YYY

[db-tv-crypt]
type = crypt
remote = db-tv:
password = XXX
password2 = YYY

A log from the command with the -vv flag

2022/08/10 22:20:01 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/10 22:44:15 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/10 22:52:35 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/10 23:09:24 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/10 23:17:21 ERROR : Media/TV/TV/Superman & Lois/Superman & Lois (2021) - S01E05 - The Best of Smallville [Bluray-1080p][AC3 2.0][DE+EN][x264]-VoDTv.m
kv: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF
2022/08/10 23:50:16 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/10 23:58:28 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/11 00:15:00 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/11 00:23:20 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/11 00:31:40 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/11 00:55:33 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout

I think you can ignore that - its a normal part of the way the change notify listener works and we should suppress that message I think.

This is a real problem, most likely networking related. However note the retry 1/10 the fact you didn't see a 2/10 likely means this was properly retried. These things happen over the internet and that is why rclone does the retries.

I've never seen that message ever in any of my logs.

It would be neat to see that mount in debug and see what other messages are printed around it.

I do get a very few sporadic ERRORs like that here/there. I tend to just ignore them as they are usually an odd network retry here/there and rarely have any impact.

Interesting...

Maybe I am wrong about that then.

@Thunder is there any sort of firewall / proxy between you and the internet which could be timing out these connections?

Could it be that this "error" is because I mounted an union remote of google and dropbox remotes?
That is because at the moment i am migrating my stuff from google to dropbox, so i have to move all the content to the new remote without making it unaccessable.
Is there a better way of migrating from google to dropbox "on the fly"?

for all the dropbox related commands i had a look on your github and used most of the commands there .

Not as I know. The Server is a Hetzner Server located in Germany.
Incoming connections to certain subdomains are prxied through cloudflare and there is a running traefik instance for the Docker containers (plex, sonarr, radarr, etc).

That error string is not in rclone nor in the go standard library - I suspect that it is being generated externally.

Searching for it leads me to believe it is very likely coming from a proxy, maybe traefik or maybe something to do with docker.

2022/08/11 14:20:57 DEBUG : /: Lookup: name="downloads"
2022/08/11 14:20:57 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2022/08/11 14:20:57 DEBUG : /: Lookup: name="downloads"
2022/08/11 14:20:57 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2022/08/11 14:21:02 DEBUG : Google drive root '': Checking for changes on remote
2022/08/11 14:21:02 DEBUG : Google drive root '': Checking for changes on remote
2022/08/11 14:21:02 DEBUG : /: Lookup: name="downloads"
2022/08/11 14:21:02 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2022/08/11 14:21:02 DEBUG : /: Lookup: name="downloads"
2022/08/11 14:21:02 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2022/08/11 14:21:07 INFO  : Dropbox root '': Change notify listener failure: upstream request timeout
2022/08/11 14:21:07 DEBUG : Dropbox root '': Checking for changes on remote
2022/08/11 14:21:07 DEBUG : Dropbox root '': Decreasing poll interval to maximum 480s

Here i got the message from above with Debug output.
Interesting is also that it seems that the dropbox remote is ignoring the --poll-intervall setting from the rclone mount as this is set to 15 seconds and according to debug it uses the default of 480 seconds.

Also it seems that the Change notify listener message appears all 8 Minutes (=480 seconds) .

There's already an issue about dropbox poller:

Dropbox --poll-interval not respected · Issue #6205 · rclone/rclone (github.com)

I'd follow @ncw's lead and go down the proxy path and see what that is possibly as either your proxy or Cloudflare is blocking/dropping something.

The Thing is Traefik is only used for the docker containers and this error message was not present when i used only Google.

cloudflare just proxies webrequests on the Domain , but rclone connection from server to Dropbox is not using any proxy

I'm only going by @ncw saying it's not from rclone which makes sense to me as I have months of dropbox logs and that message is not there for me.

I have 2 mounts running 'regular' on my Linux box and many dockers consuming those mounts. Rclone itself is not in a docker for me.

Rclone itself isnt running in a Docker container on my server .
Interesting Thing is, that it seems that everything is working fine,but I will have a look and try few things and monitor if the behaviour changes .

Different Question regarding Dropbox:
I am quite confused about API Calls. Dropbox states everywhere that there are 1 Billion API Hits allowed per Month.

Is this Limit per Team, per User, per API Connection ? Do someone with more insights now about this?

There aren't any API limits per month but there is rate limiting.

Folks have generally seen 12 TPS per second per application you have registered on the console.

I use an app reg for each mount I have.

And what is meant with 1 Billion API Calls For Transport Partners per month? If I click on Billing and review my Plan (Dropbox Business Advanced) it shows me that number.

So if I create for example 2 APIs for the same folder I would be fine?

That number is always 0:

image

You can make as many app registrations as you want. Each rclone remote can only be configured for one. You can make duplicate remotes if you wanted to split things out.

Nick, the "unexpected EOF" error happens when a video file is paused for a longer period. There is another error (i/o, I believe) that also happens frequently when pausing/stopping files. Neither error impacts playback in any way.

Ah the file was paused for about 8 Minutes, so that could be the case here.

@Animosity022 As you are the Dropbox Expert, could you make a recommendation on what to change at my mount settings?
It seems that My Dropbox Mount is a little bit slower than my Google Mount.
It is noticable at Playback start (which takes about 2 Seconds on Google and 5-7 on Dropbox) and when rewinding or skipping to a specific point in the Video (which on google is almost instant and on Dropbox takes about 3 Seconds).

I use this for my mounts:

I've noticed zero difference in Google vs Dropbox in my setup.

Both of these comments are pointing towards some external thing closing connections where no traffic has passed for a while.

In the case of the polling, dropbox uses long polling, so an http connection is held open until the response comes. If there is a change the response will come back within 15s, otherwise the connection will be held open.

What is doing the closing, I don't know though!

Quoting myself here, because last night I had the other error in my log:

ReadFileHandle.Read error: low level retry 1/10: read tcp {local IP}:56039->{remote IP}:443: i/o timeout

Just FYI. No impact on playback. Both errors always only have one retry.