Acd object not found, queue behavior & checks

Let me start by saying I never used debug prior to switching to rclone encryption so I don’t know if it was the same when using encfs. Though I know it’s not an encryption problem, rather an acd problem.

So now that I switched and am moving a ton of stuff from one acd mount to another I feel like it’s taking longer to transfer things. So I enabled -vv debug and learned about the “object not found” errors and how it and how it does all these checks. Which brings me to queue behavior.

So these transfers that don’t finish start to stack which seems to compound its self once they start stacking. Next thing I know 6 of my 8 slots are tied up waiting for checks to finish. This ultimately kills my overall speed cause sometimes files will only send at like 1/10 speed instead of using all the available speed. One time it actually filled all 8 slots with checks and nothing was moving which brings me to my 3rd point.

It seems to perform a variable number of checks, sometimes it’s 470 and others it’s more or less. At any rate, 470 checks take quite a while which leads to the second problem of stacked checks taking up queue spots. Now to the questions.

The acd thing I already know can’t be fixed. It is what it is.

So for the queue slots being used would a behavior change be worthwhile to park the transfers that are being checked and free up the queue slot?

Also, for the number of checks, is there already an option to set the number of checks? As long as it takes I would rather perform 100 checks and just reupload to save time. Would this cause a problem if/when the first file appears after the reupload finishes? In actuality, I don’t get that many actual fails. I would just rather quit and retry sooner. The number of checks would not be a problem if they were parked and opened up the queue slot.

Thanks

This has been suggested here: https://github.com/ncw/rclone/issues/1300

Unfortunately it isn’t a trivial change as it cuts across the encapsulation layers in rclone

Experiment with this option

  --acd-upload-wait-per-gb duration   Additional time per GB to wait after a failed complete upload to see if it appears. (default 3m0s)