Hi,
My first post here. Congrats to the developer(s) because you dont find such a useful tool that often.
I'm interested in speeding up downloads of splitted files. Currently I use "rclone copy" for the first chunk and "rclone cat" and an append redirection for the following ones. But "cat" is much slower than copy because it's monothread.
Could it be possible to create a copy variant for this purpose? Something like:
Thanks for your answer and reminding me about the existence of "chunker". I tried it almost two years ago and was not convinced for my use case.
Yes I split the files myself and upload the pieces, no problem there.
But when I have to download them I have to concatenate them after that and it takes time and space as the contents has grown with time.
Your suggestion makes me revisit "chunker" now, but as I'm happy with my own upload, do you think I could adapt the chunk names in the chunker directory so that the download "thinks" about them as one file? This way I wouldnt have to append them.
You can try rclone cat --files-from listing remote: > output.file where listing contains a list, in order, of all the files that you want concatenating.
The chunker backend has quite a flexible naming scheme for its chunks - check it out here
Many thanks Nick,
I think I found a way to speed up and simplify splitted downloads by using chunker's json file and modifying the naming scheme in the config as you said. I'm testing it with both copy and cat commands and looks good. The only problem with "cat" I cannot see the progress (-P) of the download as with copy (if anybody can help me, because I dont see any conclusion to a thread about this). Anyway it's no big deal as I have other ways to check whether it's going as expected.
Thanks again
Manuel F.