How to copy files without destination checks?

Help and Support
1

I have an FFL, which I am passing through the --files-from flag with --no-traverse flag. This causes an API call to the destination to check if the file exists/change or not (for each file). I want to skip this check and directly copy the file without any matching or checks.

Is there any other way to directly copy the file without checking or any flag to disable the destination check?

Thanks

2

There isn't at the moment. Those checks stop rclone copying files which are already there which save bandwidth (and is kind of the whole point of a repeatable sync).

However you aren't the first person to ask for this!

Which backend are you using? I'm guessing a bucket based store like S3/B2/GCS ?

So I could make a flag for this, say --no-check which would not check whether the destination exists and upload it anyway. Using this flag would

  • upload files that are already there wasting bandwidth
  • on some backends (eg google drive) make duplicate files
  • save the destination check when using --no-traverse

The other thing I was considering was a dedicated sync for bucket based remotes so a new command which did the copy in an optimal way.

What do you think?

3

Hi

Which backend are you using? I'm guessing a bucket based store like S3/B2/GCS ?

I am using GCS.

upload files that are already there wasting bandwidth

This will be really helpful when used --files-from flag as in most cases the diff/files_to_upload are being calculated outside the rclone scope.

4

Can please make a new issue on github about this and I'll work out how to implement it - thanks

5

I opened an issue on github. https://github.com/rclone/rclone/issues/3616
Please see.

1 Like
6

In the latest beta I fixed --files-from without --no-traverse doing a recursive scan:

I think this is likely the cause of your problem - can you give your transfer a go without --no-traverse?

7

Sure I will test the changes. Thanks.

1 Like
8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.