Jottacloud token constantly being invalidated - refresh fails

coco · May 22, 2020, 4:24pm

What is the problem you are having with rclone?

I'm using rclone and jottacloud as a restic backend.

The issue I am facing is that the jottacloud token gets invalidated very often. I cannot do the full backup, it always crashes before the end.

More precisely:

I run a new backup with restic (restic backup ...)
It starts uploading data to jottacloud
After about 1h, it crashes, and sends the following error message :

mai 22 16:25:51 coco-pc backup-restic.systemd.sh[277095]: rclone: 2020/05/22 16:25:51 ERROR : locks/17f7e8a50f30904ff71a85b5f9b227310f49f622cfacf001acd21637712e945a: Post request put error: Post https://api.jottacloud.com/files/v1/allocate: oauth2: cannot fetch token: 400 Bad Request
mai 22 16:25:51 coco-pc backup-restic.systemd.sh[277095]: rclone: Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
mai 22 16:25:51 coco-pc backup-restic.systemd.sh[277095]: rclone: 2020/05/22 16:25:51 ERROR : locks/17f7e8a50f30904ff71a85b5f9b227310f49f622cfacf001acd21637712e945a: Post request rcat error: Post https://api.jottacloud.com/files/v1/allocate: oauth2: cannot fetch token: 400 Bad Request
mai 22 16:25:51 coco-pc backup-restic.systemd.sh[277095]: rclone: Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
mai 22 16:25:57 coco-pc backup-restic.systemd.sh[277095]: Save(<index/7ea17defb0>) returned error, retrying after 391.517075ms: server response unexpected: 500 Internal Server Error (500)
mai 22 16:26:02 coco-pc backup-restic.systemd.sh[277095]: unable to refresh lock: server response unexpected: 500 Internal Server Error (500)
mai 22 16:29:40 coco-pc backup-restic.systemd.sh[277095]: rclone: 2020/05/22 16:29:40 ERROR : index/7ea17defb0b508090f2d8ca8552c4edbd6597cca16ba4f943ebc40991c8fdac0: Post request put error: Post https://api.jottacloud.com/files/v1/allocate: Post https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token: read tcp 192.168.1.15:56282->185.179.129.26:443: read: connection reset by peer
mai 22 16:29:40 coco-pc backup-restic.systemd.sh[277095]: rclone: 2020/05/22 16:29:40 ERROR : index/7ea17defb0b508090f2d8ca8552c4edbd6597cca16ba4f943ebc40991c8fdac0: Post request rcat error: Post https://api.jottacloud.com/files/v1/allocate: Post https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token: read tcp 192.168.1.15:56282->185.179.129.26:443: read: connection reset by peer
mai 22 16:30:16 coco-pc backup-restic.systemd.sh[277095]: rclone: 2020/05/22 16:30:16 ERROR : locks/48ed7b64c3657cb45db6a6fa165ed5cde6fbee5589c0cab7c775d6866c25e7fd: Post request put error: Post https://api.jottacloud.com/files/v1/allocate: oauth2: cannot fetch token: 400 Bad Request

[...]
mai 22 17:12:05 coco-pc backup-restic.systemd.sh[277095]: rclone: Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
mai 22 17:12:05 coco-pc backup-restic.systemd.sh[277095]: rclone: 2020/05/22 17:12:05 ERROR : locks/a62f980088bcedd83aa9d5eeee26643aa9f83eb8aa8e4d466e616cf6df3e0894: Post request rcat error: Post https://api.jottacloud.com/files/v1/allocate: oauth2: cannot fetch token: 400 Bad Request
mai 22 17:12:05 coco-pc backup-restic.systemd.sh[277095]: rclone: Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
mai 22 17:12:11 coco-pc backup-restic.systemd.sh[277095]: unable to refresh lock: server response unexpected: 500 Internal Server Error (500)
mai 22 17:12:16 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 716.423215ms: <lock/46fda2d632> does not exist
mai 22 17:12:22 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 931.38672ms: <lock/46fda2d632> does not exist
mai 22 17:12:29 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 1.463686923s: <lock/46fda2d632> does not exist
mai 22 17:12:36 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 2.076015618s: <lock/46fda2d632> does not exist
mai 22 17:12:43 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 1.728653694s: <lock/46fda2d632> does not exist
mai 22 17:12:51 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 3.524855795s: <lock/46fda2d632> does not exist
mai 22 17:13:00 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 7.956305759s: <lock/46fda2d632> does not exist
mai 22 17:16:45 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 10.10337179s: <lock/46fda2d632> does not exist
mai 22 17:17:01 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 18.951670903s: <lock/46fda2d632> does not exist
mai 22 17:17:26 coco-pc backup-restic.systemd.sh[277095]: Remove(<lock/46fda2d632>) returned error, retrying after 27.337308488s: <lock/46fda2d632> does not exist
mai 22 17:17:58 coco-pc backup-restic.systemd.sh[277095]: error in cleanup handler: <lock/46fda2d632> does not exist

After this, if I try to list folders, I get an error.

$ rclone lsd jottacloud:
2020/05/22 18:21:46 Failed to create file system for "jottacloud:": couldn't get customer info: Get https://api.jottacloud.com/account/v1/customer: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}

If I refresh the token, it starts working again, for about another hour, and crashes again.

What is your rclone version (output from `rclone version`)

$ rclone version
rclone v1.51.0
- os/arch: linux/amd64
- go version: go1.13.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Linux, Solus, 64 bits

Which cloud storage system are you using? (eg Google Drive)

Jottacloud

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

Initially, restic backup
After the token is invalidated, any rclone command that fetches / upload remote data doesn't work

A log from the command with the `-vv` flag (eg output from `rclone -vv copy /tmp remote:tmp`)

$ rclone -vv lsd jottacloud:
2020/05/22 18:30:08 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "-vv" "lsd" "jottacloud:"]
2020/05/22 18:30:08 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2020/05/22 18:30:08 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/22 18:30:08 DEBUG : jottacloud: Token refresh failed try 1/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
2020/05/22 18:30:09 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/22 18:30:10 DEBUG : jottacloud: Token refresh failed try 2/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
2020/05/22 18:30:11 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/22 18:30:11 DEBUG : jottacloud: Token refresh failed try 3/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
2020/05/22 18:30:12 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/22 18:30:12 DEBUG : jottacloud: Token refresh failed try 4/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
2020/05/22 18:30:13 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/22 18:30:13 DEBUG : jottacloud: Token refresh failed try 5/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
2020/05/22 18:30:14 DEBUG : jottacloud root '': Token expired but no uploads in progress - doing nothing
2020/05/22 18:30:14 Failed to create file system for "jottacloud:": couldn't get customer info: Get https://api.jottacloud.com/account/v1/customer: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}

asdffdsa · May 22, 2020, 4:40pm

hello and welcome to the forum,

do you run rclone with jottacloud on another computer or just this one computer?

coco · May 22, 2020, 4:42pm

Hi,
I run rclone only on this computer.

ncw · May 23, 2020, 10:59am

It looks like you need to run rclone config reconnect jottacloud: to get another token.

Try that and see if it works.

coco · May 23, 2020, 11:07am

This asks me to create a new token on the website, that's already what I do to make it work again (by going to rclone config).

It works again for about 1 hour, and then the token is invalidated again, and doesn't refresh

coco · May 23, 2020, 11:10am

Another thing, I run the jottacli on my laptop as well.

I tried to go in "non standard mountpoint" parameters, and it asks me to choose which device I want to use.
Two are proposed : the name of my laptop (I think this is the device used by jottacli), and Jotta.

I'm not sure which device was configured before, but which one should I choose there ?

albertony · May 23, 2020, 11:17am

which one should I choose there ?

Depends what you want. Use the special "Jotta" device if want to use rclone to access what the official clients call Sync, or Archive (these are then what is called mountpoints within the Jotta device). If you want to access what the official clients call Backup then you would want to access the device named after your computer (official client will have created a separate mountpoint for each backup folder configured).
(https://rclone.org/jottacloud/#devices-and-mountpoints)

ncw · May 23, 2020, 11:22am

It looks like Jotta have broken the oauth config by not passing on the state parameter in the oauth flow - do you see this too if you try making a new remote @albertony ?

albertony · May 23, 2020, 11:28am

I have created a new remote now. Registration, simple list commands etc works fine.

coco · May 23, 2020, 11:28am

Thanks for looking into this !

Those commands work for me as well, the only one that I noticed doesn't work is token refresh

albertony · May 23, 2020, 12:35pm

I waited until after the "expiry" time of the current token in config, and then executed a list and a small copy operation. Seems to work still. Debug log indicates it did refresh the token:

DEBUG : jotta_testing: Loaded invalid token from config file - ignoring
DEBUG : Keeping previous permissions for config file
DEBUG : jotta_testing: Saved new token in config file

coco · May 23, 2020, 1:20pm

I just created a new token and reconnected.
I will wait for expiration and execute a simple list operation, instead of running a full restic backup.

coco · May 23, 2020, 2:20pm

Same issue as before:

16:21:08 coco@coco-pc ~ rclone -vvv lsd jottacloud:                                               130 ↵
2020/05/23 16:21:19 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "-vvv" "lsd" "jottacloud:"]
2020/05/23 16:21:19 DEBUG : Using config file from "/home/coco/.config/rclone/rclone.conf"
2020/05/23 16:21:19 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/23 16:21:19 DEBUG : jottacloud: Token refresh failed try 1/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
2020/05/23 16:21:20 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/23 16:21:20 DEBUG : jottacloud: Token refresh failed try 2/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
2020/05/23 16:21:21 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/23 16:21:22 DEBUG : jottacloud: Token refresh failed try 3/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
2020/05/23 16:21:23 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/23 16:21:23 DEBUG : jottacloud: Token refresh failed try 4/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}
2020/05/23 16:21:24 DEBUG : jottacloud: Loaded invalid token from config file - ignoring
2020/05/23 16:21:24 DEBUG : jottacloud: Token refresh failed try 5/5: oauth2: cannot fetch token: 400 Bad Request
Response: {"error":"unauthorized_client","error_description":"INVALID_CREDENTIALS: Invalid client credentials"}

coco · May 23, 2020, 3:26pm

@albertony I created a new remote, waited for expiration, and tried to list files, and it worked:

17:24:29 coco@coco-pc ~ rclone -vvv lsd jottacloud2:
2020/05/23 17:24:36 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "-vvv" "lsd" "jottacloud2:"]
2020/05/23 17:24:36 DEBUG : Using config file from "/home/coco/.config/rclone/rclone.conf"
2020/05/23 17:24:36 DEBUG : jottacloud2: Loaded invalid token from config file - ignoring
2020/05/23 17:24:36 DEBUG : jottacloud2: Saved new token in config file

I will start a restic backup, and I'll tell you if the problem comes back. For now, it seems to be fixed !

Thanks again for the time you took to look into this.

coco · May 24, 2020, 9:06am

The full backup worked.

Problem is solved, though I don't know what was the issue in the first place.

Thanks again for your time.

system · May 27, 2020, 9:13am

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.