Sharepoint / one drive api / microsoft request

hello to everyone.
i sended ticket to microsoft graph api service. they request some information about my status can we set an answer together ?
i sended ticket because i receive rate limit or thorletting (generaly rate limit)
service engineer request answer for this questions.

While I am looking into your issue, I would request you to share below details so that I can check and provide the appropriate troubleshooting steps to resolve this issue: -

*** Complete Graph API request you are executing (Including Body and Headers parameters).**
*** Type of Authentication used (Application/Delegated), expired token for the same.**
*** Complete error details (Including request ID and Timestamp).**

Hi root,

I can help you with the flag to find your Graph API requests:

https://rclone.org/docs/#dump-requests

Important usage notes: ... contain sensitive info. Can be very verbose. Useful for debugging only.
Note that the bodies are buffered in memory so don't use this for enormous files.

i guess first and last question is done. Thank you Ole,
Let's check if I got it right.
when i start transfer files i have to use thoose flags. is that correct?

Correct, add the flags when you start your rclone copy sharepoint: dropbox: ... (If I remember correct)

You probably also want to add --log-file=huge_debuglog.txt - and keep an eye on memory and disk space.

wow, its generated more then 4 gb .txt files under a minute!

hi,

in the forum, onedrive is well known for heavy rate limiting, that is to be expected.
are you using onedrive personal or onedrive business?
personal is known for very heavy rate limiting, more so than business.

when you posted, there was a template of basic questions to be answered.
including the output of rclone version and the redacted config file.
the config file would show if you created a client id/secret or using the rclone default values.

hello, thank you for interested with my topic.
i am using sharepoint plan 2. i am not sure about one drive.
I'm not sure what you mean when you say config file. Are you talking about access token?

sure.

that is part of it.
you created a rclone remote to access onedrive/sharepoint.
can you post the redacted output, make sure to remove secrets such as passwords, tokens, etc..

for example, the name of my remote is sdavtest

rclone config show sdavtest:
--------------------
[sdavtest]
type = webdav
url = https://redacted-my.sharepoint.com/personal/redacted/_layouts/15/onedrive.aspx?id=%2Fsites%2Ftest01%2FShared%20Documents
vendor = sharepoint
user = redacted
pass = *** ENCRYPTED ***
--------------------

ofcourse...

[sharepoint]
type = onedrive
token = {"access_token":"xxxxx","expiry":"2022-09-21T01:57:07.4879172+03:00"}
drive_id = xxx_xx
drive_type = documentLibrary

so you are using onedrive to access sharepoint.
is that the onedrive account a personal account or business account.

for example, this is my onedrive remote.

[onedrivevb]
type = onedrive
client_id = redacted
client_secret = redacted
token = 
drive_id = redacted
drive_type = business

notice that i created my own client_id/secret and you remote is missing that.
really should create that for your remote.

https://rclone.org/onedrive/#getting-your-own-client-id-and-key
"You may choose to create and use your own Client ID, in case the default one does not work well for you.
For example, you might see throtting."

ohh this is serius status.
time is almost 4 am here. if you have time tomorrow i will like the talk with you about this. i am sure i will need your help :slight_smile:

good night, let's talk tomorrow.


i setting up like that but my config file still not similar with yours.same output as above. :frowning:
maybe i generate rclone wrong lets check it that too.

rclone config
n (for new)
name
onedrive
oauth -->enter
oauth clinet secret -->enter
1 microsoft global
advanced config -->enter
auto config -->enter
access
enter
3-sharepoint site name
done.

maybe i have to add my OAuth key manuelly ? i dont know.

https://rclone.org/onedrive/#getting-your-own-client-id-and-key

  1. create your own microsoft client id/secret
  2. create rclone remote and use that client id/secret
Microsoft App Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_id> 34ef82c0-f0f4-4e34-redacted-f2777e0b81d7

Microsoft App Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret> U4B8Q~rW3jt_dIfWQQh44mggfMFPfgredacted

Edit advanced config? (y/n)
y) Yes
n) No
y/n> n
Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine
y) Yes
n) No
y/n> y
1 Like

i understand now. i am doing.
after that i will start transfers i really wonder solituon. maybe its really work. who knows.

2022/09/27 20:51:47 Fatal error: failed to get token: oauth2: cannot fetch token: 401 Unauthorized

Response: {"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app

yes, that is a common confusion, much discussed in the forum.
need to use the Value, not the ID

here is an example,
https://forum.rclone.org/t/one-drive-client-id-and-secret/29109/2

[sharepoint]
type = onedrive
client_id = 2002382c-39a6-403c-936a-xx
client_secret = j7u8Q~7iGHCmqYhr-E7Kq8-qvRhnxx
token = {"access_token":xx"expiry":"2022-09-27T22:05:47.9685249+03:00"}
drive_id = b!NqELnHza0E6Zhc76nNGTRZEzDs_xx
drive_type = documentLibrary

its generated like that.

ok, and what is the output of
rclone lsd sharepoint: -vv

C:\Users\Administrator\Desktop\rclone-v1.59.2-windows-amd64>rclone lsd sharepoint: -vv
2022/09/27 21:14:02 DEBUG : rclone: Version "v1.59.2" starting with parameters ["rclone" "lsd" "sharepoint:" "-vv"]
2022/09/27 21:14:02 DEBUG : Creating backend with remote "sharepoint:"
2022/09/27 21:14:02 DEBUG : Using config file from "C:\Users\Administrator\AppData\Roaming\rclone\rclone.conf"
-1 2022-06-22 17:36:00 48 dd39
-1 2022-06-26 00:05:41 48 dd41
-1 2022-06-29 17:44:11 49 dd42
-1 2022-07-02 20:48:17 48 dd44
-1 2022-07-06 00:25:58 49 dd46
2022/09/27 21:14:03 DEBUG : 3 go routines active

i have no idea what is that, :slight_smile:

transfers started with 70MB per second.
we will see what will happen