Rclone move skipping over folders

Help and Support
1

What is the problem you are having with rclone?

I am moving archived folders from the cloud (Box) to a local server. I am doing slowly, only for 3 hours each night during some quiet time but I cant seem to get it to move them in an ordered manner. I would like the folders to be moved alphabetically, as in finish moving folder A before moving folder B and rclone looks like its doing that in some way but skipping over folder or doing the first subfolder and then moving onto the next top level folder without completing the previous one.
Is there a way to force it to always start at the first alphabetical folder and work from there. I am using checkers and transfers as I read that is the best approach to do it alphabetically but it doesnt seem to be working as expected

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0

  • os/version: unknown
  • os/kernel: 4.4.302+ (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.22.1
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Box

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone move Box:Archive /mnt/scratch/Archive/ --checkers 1 --transfers 1 --cutoff-mode=soft --max-duration=3h -vv --log-file=/root/scratchMove.log

ncdu of folder showing first few folders. It did the first subfolder of /2 but then moved onto /7, /10./11 and then did some subfolders of the next 3 top level folders before moving onto /18. I can confirm it does not continue those folders the next night, it just moves onto the next top level folder.

image

2

run multiple commands, one after the other, something like

  1. rclone move Box:Archive/folderA /mnt/scratch/Archive/folderA
  2. rclone move Box:Archive/folderB /mnt/scratch/Archive/folderB

not sure it will work 100%, but might also test --order-by=name --max-backlog=-1

1 Like
3

Thanks for the suggestion. You gave me an idea on how to automate this and make sure it does it alphabetically...

  1. run rclone lsd to get a list of directories
  2. pipe that into cut to get the folder name
  3. pipe that into xargs and run the move command
rclone -q lsd Box:Archive | cut -c 44- | xargs -0 -d\\n -n1 -I {} rclone move "Box:Archive/{}" /mnt/scratch/Archive --checkers 1 --transfers 1 --cutoff-mode=soft --max-duration=3h -vv --log-file=/root/scratchMove.log
4

welcome. that is very nice solution!

5

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.