I prefer not to use --max-transfer . --max-backlog does not exit.
For example, I would like to copy 7/100 at random and exit after that. Each file is 100gb, so I would like to exit after the 7th object finishes and not touch the 8th at all.
@ncw Hope you can help.
I do not believe there is a flag that sets a limit on number of files currently... (but it would admittedly probably be simply to do code in).
The best solution I could offer would be
rclone lsf /your/filepath > templist.txt
Then use a script to truncate this .txt to 7 lines (not very hard)
truncate.sh templist.txt
(you will have to make truncate, or find a command that does this out of the box)
Then finally
rclone copy /your/filepath remotename:/destinationpath --files-from templist.txt
This will result on only 7 objects (files or folder) being copied in this operation
It's a little bit of a workaround, but it's not particularly hard to pull off I think.
I'm assuming a Linux environment in my examples here, but it's just as easily doable on windows (that's where I have most experience personally).
This is not automatic as it seems. I'd need to manually locate every 7 filenames and delete them after each transfer.
Could you find a way to make it random and roll on to the next if rclone checks the files are already in destination?
Oh sorry, if you want that then you just use rclone move instead. rclone move automatically deletes files after they have successfully been transferred.
or do you see any issue with that for your use-case?
Yes, I wouldn't want to touch/edit anything from the source. Would be great if @ncw added this feature in.
It could be called --no-object 7 or something
I'd suggest instead --max-files (or --max-objects, or both depending on if we want to include folders in this). That will be more in-line with other similar flags like --max-transfer and --max-size ect.
Are you intimidated by the scripting in this? Do you want help with that? I use the --files-from method for a lot of custom-sorting stuff and it is quite reliable, so I wouldn't worry about that part of it.
Or do you prefer to wait and see if it can be integrated? It almost certainly can quite simply, but NCW has a lot of work on his to-do list so it's all down to how things get prioritized.
If you want the feature, I suggest you make a formal feature-request issue on it - so that it gets added to the list of things that need to be done - and not just forgotten in this thread:
(feel free to link this thread for context in addition to making a short description)
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.