I mounted Onedrive as a disk in my server, when I download big files, it works fine, but when I download many small files such as 100 1M-2M filses, it may comes 503 errors, I set vfs cache mode = writes, and read chunk = 5M, --tpslimit = 2, and use my own application apis, errors still happen.Is there anybody else have this problem?
What is your rclone version (output from rclone version)
latest version
Which OS you are using and how many bits (eg Windows 7, 64 bit)
centos 7.6
Which cloud storage system are you using? (eg Google Drive)
OneDrive Bussiness
The command you were trying to run (eg rclone copy /tmp remote:tmp)
I can connect it perfectly, so this is not the point
A log from the command with the -vv flag
2021/03/01 15:35:40 ERROR : One drive root 'onedrive': Statfs failed: about failed: serviceNotAvailable: Service unavailable
2021/03/01 17:19:56 ERROR : One drive root 'onedrive': Statfs failed: about failed: serviceNotAvailable: Service unavailable
Thank you for answering, after updating to the latest, problem still there, here's a full log, you can search "retry" to see where it stops, it begins fine but after downloading 1.52GB, it stops, and download the same chunk over and over again.rclone_mont.log (3.5 MB)
Checked your log and on a quick glance it looks like a file is being read as usual, then OneDrive is pacing you so everything stalls.
In my own experience Microsoft is horrible at pacing unless you are using their own client (no luck if using Linux) as even mover.io gets paced under the same rules.
Try using the following flags, in my use it helps a lot in having the applications not get paced as much:
Basically the first will limit how many transactions per second happen, this helps Microsoft avoid thinking you are trying to abuse the service, the second helps Microsoft know the application that is being used, and in theory avoid pacing as well.
Glad it worked! definitely the user-agent flag helps a lot. In my use, I've seen tpslimit also helps a bit, might be placebo, but specially when reading a lot of small files it helps keeping Microsoft happy.
Thank you very much.
I find even there is no 503 problem, MS also limits you download speed after 1GB file transfered, Is this normal? I think 1GB maybe too small.
Here's a new log, you can see at the end of this log, it begins stalls again.
The log is over 10MB, I put it in my server: http://la.imlc.tk:8880/down/p92q3ZSQJPU2