took a very quick look. and i am not all that familair with the deeper details of onedrive.
that is ok, expected output.
and in the debug log, we see that rclone was able to get a new token
DEBUG : MSFT_TEST_230826: Saved new token in config file
@asdffdsa Thanks! It still doesn't work though. Can't upload, can't download.
As far as I can tell, this "Unauthenticated" behavior and the inability to obtain a token that works represents new policies of Microsoft.
Access tokens are supposed to be created with lifetime of 60 to 90 minutes only. My tokens have been created with no lifetime at all. Refreshing should work, but it doesn't.
I'm using the free plan on Microsoft OneDrive, and maybe that's relevant. It's possible that by subscribing to "Microsoft 365" I might get better results. I'm not going to bother with that.
I find that I can still log in to Microsoft OneDrive using a browser: https://onedrive.live.com/login/
The browser interface still allows me to upload my files, so I'll just use that again.
I used rclone for well over a year and found it to be very convenient. I still use it for cloud storage on Google.
If I learn anything new or different I'll post back. Otherwise, it looks like I'm done here.
Thanks, everybody, for the help!
FYI - I have multiple free OneDrive accounts I use for testing. No issues at all - zero. I can download, I can upload.
Some have custom client_id - some have default rclone one. No difference unless I do some bulk operations then throttling kicks in. But still all works - maybe slower.
@kapitainsky : Thanks for your observations. My experience has been different, but only recently.
Have your access token(s) expired recently?
Mine did, some weeks ago. After they expired I have not been able to create a new access token that has any available lifespan. The impression I have is that a new token is born with an immediate expiration time.
It could be that Microsoft is trying to claw subscription fees out of people who previously used the free Personal OneDrive accounts. Or maybe token creation is just broken, temporarily.
I'll try again, from time to time.
Problem is I can't reproduce it. I have multiple free accounts (5GB), added one new ones last week. There is no issues with expiring tokens. All works the same as always.
Right now I have created new remote - here you are its token structure. It is the same as all other onedrive remote's tokens I use:
type = onedrive
token = {"access_token":"","token_type":"Bearer","refresh_token":"","expiry":"2023-09-04T09:30:15.349579+01:00"}
drive_id = XXX
drive_type = personal
and token is refreshed when used - e.g. after some time I can see expiry change:
"expiry":"2023-09-04T10:31:37.496375+01:00"
from my usage perspective nothing is broken or not working...
Maybe your problem is that something (firewall??) is blocking oauth token refresh? Could you test you setup from different network?
I've got exactly the same problem, but I want to add some infos:
Option region.
Choose national cloud region for OneDrive.
Choose a number from below, or type in your own string value.
Press Enter for the default (global).
1 / Microsoft Cloud Global
\ (global)
2 / Microsoft Cloud for US Government
\ (us)
3 / Microsoft Cloud Germany
\ (de)
4 / Azure and Office 365 operated by Vnet Group in China
\ (cn)
Choosing no.3 will not work, as there is no specific entry point for germany. It tries to reach https://login.microsoftonline.de/common/oauth2/v2.0/authorize but there is no microsoftonline.de, so for Germany you have to use no.1
Also using * --onedrive-region=de won't work because there is no graph.microsoft.de, the single official entry point of graph.microsoft is .com
Failed to create file system for "OneDrive:": failed to get root: Get "https://graph.microsoft.de/v1.0/drives/*****************/root": dial tcp: lookup graph.microsoft.de: no such host
EDIT:
Btw. creating a folder with the POST sample query is working here Graph Explorer | Try Microsoft Graph APIs - Microsoft Graph
The problem appears to be including the Authorization header on the PUT request for a multipart upload. From the documentation:
Including the Authorization header when issuing the PUT call may result in a HTTP 401 Unauthorized response. The Authorization header and bearer token should only be sent when issuing the POST during the first step. It should be not be included when issuing the PUT.
From my log:
2023/10/04 22:19:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/10/04 22:19:11 DEBUG : Copy-M365UserDrive.txt: Uploading segment 0/18 size 18
2023/10/04 22:19:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/10/04 22:19:11 DEBUG : HTTP REQUEST (req 0xc000523000)
2023/10/04 22:19:11 DEBUG : PUT /personal/cd3c266076684197/_api/v2.0/drives/... HTTP/1.1
Host: my.microsoftpersonalcontent.com
User-Agent: ISV|Transend Corporation|TMCTools/1.0
Content-Length: 18
Authorization: Bearer EwC...
Content-Range: bytes 0-17/18
Accept-Encoding: gzip
2023/10/04 22:19:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/10/04 22:19:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/10/04 22:19:11 DEBUG : HTTP RESPONSE (req 0xc000523000)
2023/10/04 22:19:11 DEBUG : HTTP/2.0 401 Unauthorized
Your log shows the same problem as mine. Given that including the Authorization header on the PUT request is specifically stated in the documentation as a reason for an HTTP 401 Unauthorized response, this seems like a bug.
2023/08/21 23:34:56 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/08/21 23:34:56 DEBUG : onechar.txt: Uploading segment 0/1 size 1
2023/08/21 23:34:56 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/08/21 23:34:56 DEBUG : HTTP REQUEST (req 0xc000963300)
2023/08/21 23:34:56 DEBUG : PUT /personal/1b593643b0caafe8/_api/v2.0/drives/b!IkaTZRX_Q0mPUEymmrIueaWA0RJvr-RNgPrhALokBbkNUXb6ZoZDSqJU0D1a1Qt4/items/01UH3JGVZ563EKYA2WWBB2OGA67ZTQZH72/uploadSession?guid='106e0ac8-9d58-4ced-8117-e8065bb3eff6'&overwrite=True&rename=False&dc=0&tempauth=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAvbXkubWljcm9zb2Z0cGVyc29uYWxjb250ZW50LmNvbUA5MTg4MDQwZC02YzY3LTRjNWItYjExMi0zNmEzMDRiNjZkYWQiLCJpc3MiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDAiLCJuYmYiOiIxNjkyNjc1Mjk2IiwiZXhwIjoiMTY5Mjc2MTY5NiIsImVuZHBvaW50dXJsIjoiQVlMSEFmTi9GaTRSL05vNzkwQUVxdnd4Vjk1N3lHeXJXNnRkc256eE9nUT0iLCJlbmRwb2ludHVybExlbmd0aCI6IjI4MSIsImlzbG9vcGJhY2siOiJUcnVlIiwiY2lkIjoiQVVXamJ1c1gwVStleWFiUnlJZEE5dz09IiwidmVyIjoiaGFzaGVkcHJvb2Z0b2tlbiIsInNpdGVpZCI6Ik5qVTVNelEyTWpJdFptWXhOUzAwT1RRekxUaG1OVEF0TkdOaE5qbGhZakl5WlRjNSIsImFwcF9kaXNwbGF5bmFtZSI6IkdyYXBoIiwiYXBwaWQiOiIwMDAwMDAwMy0wMDAwLTAwMDAtYzAwMC0wMDAwMDAwMDAwMDAiLCJ0aWQiOiI5MTg4MDQwZC02YzY3LTRjNWItYjExMi0zNmEzMDRiNjZkYWQiLCJ1cG4iOiJyaWNoYnJhbmR0QGNvbWNhc3QubmV0IiwicHVpZCI6IjAwMDNCRkZEMDkxNUY1OTIiLCJjYWNoZWtleSI6IjBoLmZ8bWVtYmVyc2hpcHwwMDAzYmZmZDA5MTVmNTkyQGxpdmUuY29tIiwic2NwIjoibXlmaWxlcy5yZWFkIGFsbGZpbGVzLndyaXRlIGFsbHByb2ZpbGVzLnJlYWQiLCJ0dCI6IjIiLCJpcGFkZHIiOiIyMC4xOTAuMTM1LjQ3In0.wDTZB0GqNpQOc_ovoqi8lmmEJq8OM06W8CskitmJfsA HTTP/1.1
Host: my.microsoftpersonalcontent.com
User-Agent: rclone/v1.63.1
Content-Length: 1
Authorization: XXXX
Content-Range: bytes 0-0/1
Accept-Encoding: gzip
2023/08/21 23:34:56 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/08/21 23:34:57 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/08/21 23:34:57 DEBUG : HTTP RESPONSE (req 0xc000963300)
2023/08/21 23:34:57 DEBUG : HTTP/2.0 401 Unauthorized
Ultimately, the failure caused by the inclusion of the Authorization header on the PUT request is due to the upload operation hitting different backends. The POST to create the upload session is against OneDrive, but the PUT to upload the data is against Graph. Each of these has its own token, and they cannot be used interchangeably. For Graph, the required token is returned in the POST response in the tempauth field. When making the PUT request, both tokens are supplied, and the Authorization header is given precedence in the failing scenarios.
Here's a sample where I have replaced the actual tokens with ONEDRIVE_TOKEN and GRAPH_TOKEN for clarity.
2023/10/04 22:19:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/10/04 22:19:10 DEBUG : Copy-M365UserDrive.txt: Starting multipart upload
2023/10/04 22:19:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/10/04 22:19:10 DEBUG : HTTP REQUEST (req 0xc000754900)
2023/10/04 22:19:10 DEBUG : POST /v1.0/drives/cd3c266076684197/items/CD3C266076684197!163:/Copy-M365UserDrive.txt:/createUploadSession HTTP/1.1
Host: graph.microsoft.com
User-Agent: ISV|Transend Corporation|TMCTools/1.0
Content-Length: 116
Authorization: Bearer ONEDRIVE_TOKEN
Content-Type: application/json
Accept-Encoding: gzip
{"item":{"fileSystemInfo":{"createdDateTime":"2023-06-30T14:05:12Z","lastModifiedDateTime":"2023-06-30T14:05:12Z"}}}
2023/10/04 22:19:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/10/04 22:19:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/10/04 22:19:11 DEBUG : HTTP RESPONSE (req 0xc000754900)
2023/10/04 22:19:11 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Cache-Control: no-store, no-cache
Client-Request-Id: f561e6ea-235e-450f-bb5b-ed434c7b97f7
Content-Type: application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8
Date: Thu, 05 Oct 2023 03:19:09 GMT
Location: https://graph.microsoft.com
Odata-Version: 4.0
Request-Id: f561e6ea-235e-450f-bb5b-ed434c7b97f7
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"South Central US","Slice":"E","Ring":"5","ScaleUnit":"005","RoleInstance":"SN4PEPF000006F6"}}
614
{"@odata.context":"https://graph.microsoft.com/v1.0/$metadata#microsoft.graph.uploadSession","expirationDateTime":"2023-10-05T03:34:09.903Z","nextExpectedRanges":["0-"],"uploadUrl":"https://my.microsoftpersonalcontent.com/personal/cd3c266076684197/_api/v2.0/drives/b!FvAIvh0_20O_ObJZ91ioW810sFI9iN1CjQCdmf-IB-TwYMbbJO_BTZdmFk7EGcR9/items/01WIFDVRR7FXWDZQLMENG3IFGFOWYBL35W/uploadSession?guid='57535ede-8024-4c47-a5c9-02ddd4978c74'&overwrite=True&rename=False&dc=0&tempauth=GRAPH_TOKEN"}
0
2023/10/04 22:19:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2023/10/04 22:19:11 DEBUG : Copy-M365UserDrive.txt: Uploading segment 0/18 size 18
2023/10/04 22:19:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2023/10/04 22:19:11 DEBUG : HTTP REQUEST (req 0xc000523000)
2023/10/04 22:19:11 DEBUG : PUT /personal/cd3c266076684197/_api/v2.0/drives/b!FvAIvh0_20O_ObJZ91ioW810sFI9iN1CjQCdmf-IB-TwYMbbJO_BTZdmFk7EGcR9/items/01WIFDVRR7FXWDZQLMENG3IFGFOWYBL35W/uploadSession?guid='57535ede-8024-4c47-a5c9-02ddd4978c74'&overwrite=True&rename=False&dc=0&tempauth=GRAPH_TOKEN HTTP/1.1
Host: my.microsoftpersonalcontent.com
User-Agent: ISV|Transend Corporation|TMCTools/1.0
Content-Length: 18
Authorization: Bearer ONEDRIVE_TOKEN
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
I have posted a potential fix for this here
Please reply on the issue if you have comments.
1 Like