Move from Sync to Async

Help and Support
1

Im using Dropbox and uploading a lot of small files, currently im using sync but want to change it with async as recommended for lot of small files, normally my file size is less then 1MB, below is my current mount cmd

rclone mount Dropbox: /www/wwwroot/videos/box --daemon --allow-other --poll-interval 10s --tpslimit 12 --tpslimit-burst 12 --dir-cache-time 5000h --vfs-cache-mode full --vfs-cache-max-age 150h --vfs-cache-max-size 3000G --cache-dir /www/wwwroot/cache --transfers 500 --log-file /www/wwwroot/rclone.log --dropbox-batch-mode sync

when i will change --dropbox-batch-mode sync with --dropbox-batch-mode async should i have to remove --transfers 500 ?
please also guide what other changing i do needed? i aware to run rclone config and select async from there and also batch size set to 1000, any other change that needed?
Where i need to add this rclone check in mount cmd or some where in rclone config?
Thanks

2

No, but I think --transfers 500 is too high, I'd recommend --transfers 64 to start with.

No, you can put the flags on the rclone mount command line and nothing else is needed.

That is when you are doing a sync and want to check that it all worked. I don't think you can use it with rclone mount.

3

Im using --transfers 500 for last few weeks and there is no error on my log.txt file,
but as you recommended i can reduce it with 250?

so my mount cmd look like below

rclone mount Dropbox: /www/wwwroot/videos/box --daemon --allow-other --poll-interval 10s --tpslimit 12 --tpslimit-burst 12 --dir-cache-time 5000h --vfs-cache-mode full --vfs-cache-max-age 150h --vfs-cache-max-size 5000G --cache-dir /www/wwwroot/cache --transfers 250 --log-file /www/wwwroot/rclone.log --dropbox-batch-mode async

Thanks @ncw

4

Actually this is unlikely to matter too much for a mount where uploading 250 or 500 files at once is very unlikely to happen unless you copy in logs of small files very quickly.

If it is working, then good!

5

My file size are less then 1MB, does this make any sense?

6

I think your settings look ok for this, so Id give it a go and watch out for errors in the logs.

7

Im extremely sorry its not mount its move, plz check my final move cmd and guide me further if necessary

rclone move /www/wwwroot/videos/encode-pinoy/video Dropbox:pinoy --delete-empty-src-dirs --tpslimit 12 --tpslimit-burst 12 --transfers 250 --log-file /www/wwwroot/rclone.log --dropbox-batch-mode async

can i add --dropbox-batch-size in cmd?
i run rclone config and select batch mod from advance setting, amything else that can help me to move file quickly and safely?
Thank

8

Yes you can ..

Or you can do that.

You'll have to experiment to find the optimum values.

9

do i still need --transfers 250 ?

10

Try it out and see if it works for you.

Do you want to transfer 250 files at a time? What's the performance like with that on? Does it do what you want?

11

i was using transfer 250 with sync cmd and it was working fine, i have to move a lot of small files, less then 1mb file size, now i read that its is recommended async for small files, so i change my move cmd with async, and i guess when we add batch size we dont need transfers flag as it sound same thing but not sure,

12

Test it out and see how it performs.

13

Finally i will go with sync mod and transfers 250, this is much faster then async, my log file is empty means there is no moving error,
Thanks for help

1 Like
14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.