--no-traverse explanation please

Help and Support
1

here is the doc link

The --no-traverse flag controls whether the destination file system is traversed when using the copy or move commands. --no-traverse is not compatible with sync and will be ignored if you supply it with sync.

If you are only copying a small number of files (or are filtering most of the files) and/or have a large number of files on the destination then --no-traverse will stop rclone listing the destination and save time.

However, if you are copying a large number of files, especially if you are doing a copy where lots of the files under consideration haven't changed and won't need copying then you shouldn't use --no-traverse.

See rclone copy for an example of how to use it.

example case: lets say I running a copy command folder1 to folder2, in this case if I use --no-traverse will all file again reupload (copy) and replace the older time file copied?

currently rclone check filename and file size, I think may be md5 hash too. if match rclone won't upload that file.

2

No. Only files which need to be transferred will be uploaded

With --no-traverse rclone does not list all destination but checks files one by one to determine if transfer it or not. One by one is slow but can be still faster for small transfers to destinations with huge amount of objects.

If you want to upload everything regardless if present at the destination or not you can use --no-check-dest

1 Like
3

sorry my english is not good. lets say I want to upload new/abc.txt and --no-traverse will check only if remote folder new/abc.txt is available or not. did you mean that?

actually I have like 20K~30K small text files. 100~200KB each file size. file is saved in my local hard disk. so i run copy command to cloud server just for a back up.

now problem is it takes a way long time since the number fo file is bigger.

i tried --no-traverse with max age 24 hour. it took like 5~10min max. then I removed --no-traverse and currently it's running almost 50min passed still it's running. so I am not fully clear about the --no-traverse command.

rclone copy --max-age 24h --no-traverse /path/to/src remote:
4

Yes. It will only check this one file.

In general I would say if not sure then do not use random flags you just find.

Keep things simple. You want to copy your files just run:

rclone copy /path/to/src remote:

Small files transfers are always much slower than big files. Nothing you can do about it.

--no-traverse makes sense when you have 10 files to copy to destination with 10k files. There is no exact number to tell you when it makes sense to use it. In general very few files to destination with huge number of files.

Of course feel free to experiment to see what works for you and what not.

If you need advice how to make you operation faster post all details and maybe somebody can suggest some improvements.

1 Like
5

without --no-traverse how does rclone copy work? if I already set a path where to upload why would rclone check all the folders and then upload?

i don't use random flag without understanding. after fully confirming then I try. it was just for testing purpose. copied a demo folder and then tried it.

6

Without --no-traverse rclone will list all files from destination to determine what to transfer.

Copy operation itself will have the same final result - with or without --no-traverse.

Difference is that in some very specific situations --no-traverse can be faster. In other cases it will be slower.

7

i tested again i set 2hour max age and --no-traverse it took 30sec to complete the copy command. and without --no-traverse almost 5 min passed. copy command is still running.

8

Yes. Because by setting --max-age you most likely transfer much less files. Maybe only few. Which is good case for --no-traverse.

But rest of your files is not copied.

9

--max-age without --no-traverse takes a long time
--max-age with --no-traverse takes less than a min to complete the copy process.

10

run:

rclone ls --max-age 2h /path/to/src

It will tell you how many files you copy.

If it is only few and your destination has already many files then indeed --no-traverse will be faster.

If this is what you want - just copy few files and ignore everything else.

11

sorry for bothering you.

my question is. if copy command is already using default --no-traverse how come for --max-age without --no-traverse takes way too longer time?

and with --no-traverse flag it takes less than a min?

12

There is no default --no-traverse.

Using max-age you only ask to copy few files. So in your specific situation - where probably there are already many files at the destination adding --no-traverse makes it faster.

13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.