asari-fd
(Takashi Asari)
July 11, 2018, 6:16am
#1
I have to unarchive some large archive, so I want to do something like:
rclone cat --files-from \
<(echo path1/to/archive.tar.gz.aa;
echo path2/to/archive.tar.gz.ab; ... ) \
remote: | \
/some/decrypt/cmd --with-key /path/to/key | \
tee >(time md5sum) >(tar -zxv) >/dev/null
… but in fact the order of cat
seems somewhat random. Are there any ways to have rclone respect the order of files-from
argument?
I’m using rclone v1.42.
ncw
(Nick Craig-Wood)
July 11, 2018, 1:33pm
#2
Hmm, I suspect rclone is doing them in parallel.
Try --checkers 1
I think that will turn off the parallelism and get you an in order list.
asari-fd
(Takashi Asari)
July 12, 2018, 4:52am
#3
Thanks for your reply. I tried:
rclone cat --checkers 1 --files-from <(echo path1/to/A; echo path2/to/B) remote:path
and
rclone cat --checkers 1 --files-from <(echo path2/to/B; echo path1/to/A) remote:path
but I got the same results.
( I have managed to unarchive the large archive ( unarchivecmd < <(rclone cat ...; rclone cat ...; rclone cat ...)
), so please never mind about that. I still believe that ordered cat
would be helpful though. )
ncw
(Nick Craig-Wood)
July 12, 2018, 12:58pm
#4
I’ve just tried to validate your results, and I haven’t been able to - I find cat always outputs things in order.
Can you show me how to reproduce your result? Preferably with some CLI commands I can cut and paste!
Thanks
asari-fd
(Takashi Asari)
July 13, 2018, 3:12am
#5
Sure. I’m using Google Drive (Team Drive), so can I invite you to our testing (sandbox) drive?
$ rclone --version
rclone v1.42-005-g56e1e820β
- os/arch: linux/amd64
- go version: go1.10.1
$ rclone cat --checkers 1 --files-from <(echo A.txt; echo B.txt) fd-public:
A
B
$ rclone cat --checkers 1 --files-from <(echo B.txt; echo A.txt) fd-public:
A
B
$ rclone cat --checkers 1 --files-from <(echo B.txt) fd-public:
B
ncw
(Nick Craig-Wood)
July 14, 2018, 10:41am
#6
Thanks for that, I see where I’ve got confused. Rclone will always output things in order, in alphabetical order, not the order you put things in the --files-from file.
asari-fd
(Takashi Asari)
July 17, 2018, 1:29am
#7
Oh I see. Thanks again for your reply, and sorry for my bad explanation.
Now I understand that I can use alphabetically ordered paths when I use --checkers 1
. It’s non-random, which is maybe not the best, but great.
1 Like