How to move only files that are not already being moved by another rclone process? I have 2 separated lines of internet and want to move files from the same folder into 2 different clouds

Help and Support
1

STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.

How to move only files that are not already being moved by another rclone process? I have 2 separated lines of internet and want to move files from the same folder into 2 different clouds

rclone v1.64.0

  • os/version: Microsoft Windows 10 Pro 22H2 (64 bit)
  • os/kernel: 10.0.19045.3516 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.21.1
  • go/linking: static
  • go/tags: cmount
Yes

Which cloud storage system are you using? (eg Google Drive)

Google drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

c:\rclone\rclone.exe move --bind=192.168.1.2 --ignore-existing --verbose --transfers 4 --checkers 8 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 --stats 1s --max-depth 1 S:\ googledrive1:

and

c:\rclone\rclone.exe move --ignore-existing --verbose --transfers 4 --checkers 8 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 --stats 1s --max-depth 1 S:\ googledrive2:

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[googledrive1]
type = drive
scope = drive
token = XXX
team_drive = XXX
root_folder_id = XXX
client_id = XXX
client_secret = XXX

[googledrive2]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = XXX

A log from the command that you were trying to run with the -vv flag

the same files will be moving altogether on both nics

How do I move only files that are not already being moved by another rclone process? I have 2 separated lines of internet and want to move different files from the same source folder into 2 different clouds simultaneously. Files in the source folder are being generated continuously. I tried min-age but most of the times both rclone processes are moving the same files, might be because when the first rclone process finished with some files it then picks another lof of files that, as the time passed, fall into the same min-age filter. max-age also couldnt solve this because the second process sometimes busy with uploading and thus the files passed through the filter. Rclone also doesnt lock the files when moving so I cant use any other external command to handle this.

2

You could apply some naming convention logic maybe? all files starting with A go to destination1 and all files starting with B to destination2.

3

When I did that, I did check-first with one running order by ascending and the other running descending.

4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.