OneDrive JWT issue

Suspected Bug
1

What is the problem you are having with rclone?

rclone works for OneDrive initially, but after about an hour of inactivity stops working with the following error:

Failed to create file system for "OneDrive:": failed to get root: InvalidAuthenticationToken: IDX14100: JWT is not well formed, there are no dots (.).
The token needs to be in JWS or JWE Compact Serialization Format. (JWS): 'EncodedHeader.EndcodedPayload.EncodedSignature'. (JWE): 'EncodedProtectedHeader.EncodedEncryptedKey.EncodedInitializationVector.EncodedCiphertext.EncodedAuthenticationTag'.

Seems to be the same issue as https://forum.rclone.org/t/onedrive-token-error-after-30-minutes/40648 which is closed without an adequate fix

It is happening when the initial token expires. I'm certainly no expert here, but I think it's related to passing the token incorrectly - see https://learn.microsoft.com/en-us/answers/questions/1382499/invalidauthenticationtoken-with-https-graph-micros

I'd appreciate any help!

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.2
- os/version: debian trixie/sid (64 bit)
- os/kernel: 6.6.15-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Microsoft OneDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone ls OneDrive:

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[OneDrive]
type = onedrive
token = XXX
drive_id = XXX
drive_type = personal
delta = true
fast-list = true

A log from the command that you were trying to run with the -vv flag

~$ rclone ls OneDrive: -vv
<7>DEBUG : rclone: Version "v1.65.2" starting with parameters ["rclone" "ls" "OneDrive:" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "OneDrive:"
<7>DEBUG : Using config file from "/home/mike/.config/rclone/rclone.conf"
Failed to create file system for "OneDrive:": failed to get root: InvalidAuthenticationToken: IDX14100: JWT is not well formed, there are no dots (.).
The token needs to be in JWS or JWE Compact Serialization Format. (JWS): 'EncodedHeader.EndcodedPayload.EncodedSignature'. (JWE): 'EncodedProtectedHeader.EncodedEncryptedKey.EncodedInitializationVector.EncodedCiphertext.EncodedAuthenticationTag'.
2

welcome to the forum,

fast-list = true is a global flag. as such, cannot go into the config file.
add that to the command.

3

Thanks for that re fast-list - I've fixed that.
Removing that from config doesn't fix this issue though.

4

correct.

as per rclone docs, any reason not to create your own client id/secret?
as you are using rclone default, which is shared with all other users not creating the client id/secret.

5

As per rclone docs "You may choose to create and use your own Client ID, in case the default one does not work well for you. For example, you might see throttling."

The docs imply to me that creating your own Client ID is usually unnecessary. I was planning to only create my own ID if was getting issues with throttling. However I will do this and see if it makes a difference with this jwt issue - I'm not sure it will.

6

I think it actually states the opposite as throttling is generally a bad thing but if it's confusing perhaps it could be more clear. I know the Google Drive one is mega clear as I wrote it :slight_smile:

7

Onedrive is even worse than gdrive with throttling. But yes probably nothing to do with your issue.

Moving forward I would though suggest to do it - you will quickly discover it yourself.

8

fwiw, create a new remote, not to re-use the other one.

9

I've now created my own Client ID and used that successfully yesterday.

However the issue persists, as today the "token needs to be in JWS or JWE Compact Serialization Format. (JWS)" errror has returned.

Any thoughts?

10

well, for sure, i do not know what is going on.
maybe someone else does?

in the mean time, i would start over, with a very simple script
when the issue returns, now you have a the complete history of tokens and debug logs.
each time you run rclone, do the following:

  1. save rclone.conf
  2. run rclone cmd using --dump=headers --retries=1 and save all the logs

also, there are other --dump flags that might yield useful info.
i do not use oauth2 remotes much, so not sure which flags, if any, to try

11

Have you tried to reauthenticate your account?

What OS are you using? Wonder if some locale or encoding is messing up the token as otherwise it would be more widespread problem.

12

This is a strange error! Rclone just gives the token that it gets from onedrive back it doesn't manipulate the token in any way.

There are two things worth investigating in the thread you linked

I'm getting the same error when using a token to access the MS Graph API, but it only occurs 60 minutes after the first token is received. It appears Microsoft Graph API is returning this error when my token has expired rather than what I expected, an 80049228 error code.

To fix this, I simply assume the above error is an expiration error and use the stored refresh token to grab another access token. It's quite odd, as the tokens work fine for the first hour. Once they're expired, I'd expect a better error message than this.

So is this just an oddly spelled expiration error? If so rclone won't be recognising it and refreshing the token.

Issue has been resolved when I was passing the token wrong as "Bearer eydghdffgfdg...." space between the Bearer and token removed and issue got solved. I don't know will be the same for others.

Sounds unlikely but investigating exactly how it is passed would be a good idea. Use -vv --dump headers,auth to see the Authorization headers (but don't post them online!)

1 Like
13

Thanks for your comments so far.

This bug is really bugging me!! I had gone a few days without it resurfacing and was starting to think it had magically gone away, but now it's back. I can't find any consistent time that it occurs. It is always after I have not used rclone for a while, so I have assumed it is related to a refresh token, but it isn't consistent - often I have started up after many hours and the error hasn't happened.

To jcastill: Yes, reauthenticating the account works and gets rclone working again. Only for the error to recur at some stage down the track.

I have run the command with dump headers, auth as requested:

rclone ls OneDrive: -vv --dump headers,auth

My current rclone.conf:

[OneDrive]
type = onedrive
client_id = {redacted}
client_secret = {redacted}
token = {"access_token":"EwCAA8...{redacted}...$","expiry":"2024-03-04T21:37:45.971345782+11:00"}
drive_id = ba6de146bd54b21e
drive_type = personal
user-agent = "ISV|rclone.org|rclone/v1.65.2"

The output:

<7>DEBUG : rclone: Version "v1.65.2" starting with parameters ["rclone" "ls" "OneDrive:" "-vv" "--dump" "headers,auth"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "OneDrive:"
<7>DEBUG : Using config file from "/home/mike/.config/rclone/rclone.conf"
<7>DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
<7>DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
<7>DEBUG : HTTP REQUEST (req 0xc0008d6700)
<7>DEBUG : GET /v1.0/drives/ba6de146bd54b21e/root HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.65.2
Authorization: Bearer EwCAA8l...{rest of bearer token redacted}...=
Accept-Encoding: gzip

<7>DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
<7>DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
<7>DEBUG : HTTP RESPONSE (req 0xc0008d6700)
<7>DEBUG : HTTP/2.0 401 Unauthorized
Client-Request-Id: {redacted}
Content-Type: application/json
Date: Mon, 04 Mar 2024 05:52:08 GMT
Request-Id: {redacted}
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
Www-Authenticate: Bearer realm="", authorization_uri="https://login.microsoftonline.com/common/oauth2/authorize", client_id="{redacted}"
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"Australia Southeast","Slice":"E","Ring":"4","ScaleUnit":"001","RoleInstance":"ML1PEPF0000E51D"}}

<7>DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Failed to create file system for "OneDrive:": failed to get root: InvalidAuthenticationToken: IDX14100: JWT is not well formed, there are no dots (.).
The token needs to be in JWS or JWE Compact Serialization Format. (JWS): 'EncodedHeader.EndcodedPayload.EncodedSignature'. (JWE): 'EncodedProtectedHeader.EncodedEncryptedKey.EncodedInitializationVector.EncodedCiphertext.EncodedAuthenticationTag'.

Thanks in advance for looking into this.

14

One thing that occurs to me... Rclone does it's refresh of the token on a time basis which means your computer clock has to be set properly, both time and timezone. This is unlikely to be the problem but worth checking.

Otherwise the request and response looks normal. I think the best assumption is to think that "The token needs to be in JWS or JWE..." Really means that the token has expired.

15

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.