Dropbox supplying tokens with very short life

Help and Support
1

Hello.

What is the problem you are having with rclone?

I have set up rclone to connect to a Dropbox account on a headless Linux system earlier today. I obtained the auth token using using the Windows version on a Windows 10 PC using Firefox.

However the token provided by Dropbox had a very short life of only four hours.

{"access_token":"xxx","token_type":"bearer","refresh_token":"xxx","expiry":"2024-02-25T19:12:00.9935995Z"}

Because of this the sync job I left running ran for a while then threw up lots of these failures:

Failed to copy: failed to open source object: Post "https://content.dropboxapi.com/2/files/download": couldn't fetch token: invalid_grant: maybe token expired? - try refreshing with "rclone config reconnect dropbox:"

I have done so, gone through the steps, and pasted the new token. However looking in the rclone.conf file the new token also only has a four hour expiry:

...,"expiry":"2024-02-26T00:18:54.3264971Z"}

I wanted to set this up as a scheduled job to run each night but obviously this is not possible if I can't get a token with a reasonable life.

I do not know whether it is an issue with rsync, Dropbox, or something with our specific account.

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.2
- os/version: centos 7.9.2009 (64 bit)
- os/kernel: 3.10.0-1160.108.1.el7.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Dropbox (business account)

I have also created our own Dropbox App ID.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync dropbox: /data/dropbox --fast-list --check-first --tpslimit 10 --tpslimit-burst 0 --progress

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[dropbox]
type = dropbox
token = XXX
client_id = XXX
client_secret = XXX

A log from the command that you were trying to run with the -vv flag

I do not have this because I was not expecting it to fail. I have not yet rerun the sync because there is no point until I can get a token that lasts more than four hours.

I welcome any suggestions and ideas please.

Thank you,
Paul

2

Token expiry time is irrelevant as it normally refreshes automatically when needed.

It would indicate that there is some other issue with your setup. Please post full DEBUG log which should shed some light on it.

You do not have to run full sync - it would generate rather big log. Just wait for your token to expire and try to transfer some small test data capturing required log.

1 Like
3

Hello,

Thank you for explaining how it should work, that's very useful and makes sense.

It seems to have worked correctly this time, without me changing anything:

[root@xxx ~]# rclone sync dropbox:zzz-test/ /data/test --fast-list --check-first --tpslimit 10 --tpslimit-burst 0 -vv
2024/02/26 15:17:31 INFO  : Starting transaction limiter: max 10 transactions/s with burst 1
2024/02/26 15:17:31 DEBUG : rclone: Version "v1.65.2" starting with parameters ["rclone" "sync" "dropbox:zzz-test/" "/data/test" "--fast-list" "--check-first" "--tpslimit" "10" "--tpslimit-burst" "0" "-vv"]
2024/02/26 15:17:31 DEBUG : Creating backend with remote "dropbox:zzz-test/"
2024/02/26 15:17:31 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/02/26 15:17:31 DEBUG : dropbox: Loaded invalid token from config file - ignoring
2024/02/26 15:17:31 DEBUG : Saving config "token" in section "dropbox" of the config file
2024/02/26 15:17:31 DEBUG : dropbox: Saved new token in config file

It then completed syncing the small test folder successfully.

When it failed previously, it was part way through a large sync when it passed the expire time. Does rclone only check the token at the start of a process, or should it also fetch a new token if required part way through a process?

Once my full sync is completed this shouldn't be a problem anyway because it will only run once a day, so will always need to fetch a new token first, and will take a lot less than 4 hours to complete.

Thank you again for your help and for a great project.

Paul

1 Like
4

Anytime it is needed to authenticate transaction. It can happen many times during single rclone sync operation. It also means that your rclone.conf file must be writeable. For remote like dropbox updated tokens will be written there all the time.

1 Like
5

Makes sense. It's strange that it didn't work previously but did now. I'll see how it goes and update here as necessary.

rclone.conf is readable and writeable by root, and it is root that runs the process.

-rw------- 1 root root 409 Feb 26 15:46 rclone.conf

Thank you.

6

After running every night for a week this is continuing to work correctly. I still don't know what caused the original issue but it's all fine now.

Thanks again for your help and explanations.

7

Make sure that your server has correct date and time... as if it drifts e.g. due to not working ntp it can cause all sort of issues with expiry times.

1 Like
8

Good point, thank you.

The date and time were roughly correct (within a couple of minutes) but the NTP service was not set up or running. Now installed and running, and showing the correct date and time.

I will check it remains on UTC at the end of the month when we switch to BST instead of GMT (I'm in the UK), and adjust NTP settings if necessary, as I assume the tokens are based on UTC.

9

OK. This is far from correct:) and could indeed cause issues. Good blog post here - 10 seconds is considered acceptable.

You do not have to adjust anything if all setup correctly. working NTP should always keep your computer within tens of milliseconds from reference time.

What time zone you are using to present it is effectively just cosmetics - to adjust what is displayed to human operator local preference:) which for you will be Europe/London. It takes into account daylight saving changes etc. as long as your tz database is correct. But for UK it is the same since time immemorial.

1 Like
10

That may well have been the cause of the problem then.

Thank you again, this has been really helpful.

1 Like
11

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.