Default --onedrive-chunk-size does NOT work

Suspected Bug
21

Hello,
I 've the same kind of 504 proxy error when using rclone to onedrive.
(onedrive support propose me to contact rclone forum)

Is there any results of your analysis and investigations so far, please ?

Thanks,
gilhuib

22

Have you got a very slow internet connection? If so you will need to lower --onedrive-chunk-size from the defalt of 10M. Try 5M , 2560k, 1280k, 640k, 320k in that order and stop when it works!

23

Yes :slight_smile:

The problem observed is that my synchronisation to onedrive stucked after a while.

My analysis is that data are uploaded (i see it on my upload bandwith occupation when rclone runs) and the visible errors in log like:
2020/12/01 21:01:31 DEBUG : Couldn't decode error response: EOF
2020/12/01 21:01:31 DEBUG : pacer: low level retry 5/10 (error 504 Gateway Timeout: )
2020/12/01 21:01:31 DEBUG : pacer: Rate limited, increasing sleep to 2s

I tried to synchronize different set of data with different set of options, same result. For instance :

  • rclone sync -P $LOCAL_PATH_TO_SYNC $REMOTE_PATH --exclude $FILTER
  • rclone sync -P $LOCAL_PATH_TO_SYNC $REMOTE_PATH --exclude $FILTER --log-file=/var/log/rclone2ondrive.sh.log --cache-db-purge --contimeout=5m --onedrive-chunk-size 50M -vv --dump responses
  • rclone sync -P $LOCAL_PATH_TO_SYNC $REMOTE_PATH --exclude $FILTER --log-file=/var/log/rclone2ondrive.sh.log --contimeout=10m --onedrive-chunk-size 5M -vv --dump responses
  • rclone sync -P $LOCAL_PATH_TO_SYNC $REMOTE_PATH --exclude $FILTER --log-file=/var/log/rclone2ondrive.sh.log --contimeout=10m --onedrive-chunk-size 5M --tpslimit 1 -vv --dump responses

My understanding was that the 504 is generated by onedrive backend.
I've no idea of any technical limitation on onedrive part (no huge files, just personal pictures and videos).
I've explain my problem to MS support. They proposed me to start discussion with rclone team.

--- Setup ---
Network : DSL Connection 16M/1M / standard router / no proxy on home network
rclone v1.53.2
- os/arch: linux/arm
- go version: go1.15.3
Data sync from a NAS to OneDrive Premium account

24

Can you try the smaller values I suggested above? I think at some point it should start working!

--onedrive-chunk-size 5M
--onedrive-chunk-size 2560k
--onedrive-chunk-size 1280k
--onedrive-chunk-size 640k
--onedrive-chunk-size 320k
25

Sounds good with "--onedrive-chunk-size 1280k" option.
Thanks a lot.

Did I miss some info somewhere on the OneDrive limitations/specififications or on the way to use rclone ?

26

I've recently (only in the latest beta) put a warning about this into rclone

1 Like
27

:ok_hand:
Thanks again.

28

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.