Rclone Syn question

Help and Support
1

I want to syn two folder. my source folder has already 10K files approx. if I run syn command after 100% syn, if there is a new file in source file will rclone check all 10K files again?

2

Yes, it will as there's no cache or remembering in a sync command. It's a one time deal.

3

basically i need to keep syn. is there anyway I can create a cache or something? each time checking all file will be time consume.

4

Unsure as I'm not familiar with what OS you are running, more details on the use case, what version you are running, etc.

5

sorry. i using windows OS. i want to syn source folder: google drive and destination folder: FTP host

6

We can probably make it quick, but we need to know your situation better to do so.

Therefore, please post the redacted output from these commands:

rclone version
rclone config show yourDrive:
rclone config show yourFTP:
rclone sync --dry-run --log-file=rclone_log.txt --progress yourDrive:folder/to/sync yourFTP:folder/to/sync
7

this is the output result

rclone sync --dry-run --log-file=rclone_log.txt --progress "Gdrive:test" "FTP:test"
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            1 / 1, 100%
Elapsed time:          0s

and this is the log

2022/10/03 11:53:47 NOTICE: New Text Document.txt: Not copying as --dry-run
8

I miss the redacted output from each of these:

rclone version
rclone config show GDrive:
rclone config show FTP:

and then I would like to see the output from this using your actual data (with 10K files) in GDrive:

rclone sync --dry-run --log-file=rclone_log.txt --progress GDrive:actual/folder/to/sync GDrive:actual/folder/to/sync

I don't need the log, just the stats at the end.

Have you tried transferring the 10K files to your FTP destination? or have another folder with 10K files in your FTP?

9

sorry i havn't test 10K files. i am going to start a project which will have that much of files. it's actually more than 30K files so far.

i tried to syn around 3.5K files. once syn complete, next run took around 135 second.

rclone version is: 1.59.2

I am not quite sure about this info

rclone config show GDrive:
rclone config show FTP:

what info are you looking for? can you please tell me what command line should I run to get your desired output?

10

Please let me see the full output from this command:

rclone version

You can execute it just like the rclone sync you executed above.

11
12

Thanks, please post the output of:

rclone version

again, but this time in a code block similar this:

This makes it easier to find and read.

13 
rclone v1.59.2
- os/version: Microsoft Windows Server 2016 Datacenter Evaluation 1607 (64 bit)
- os/kernel: 10.0.14393.5291 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.18.6
- go/linking: static
- go/tags: cmount
14

Thanks, now the same for:

rclone config show GDrive:

and

rclone config show FTP:

Please redact any secrets (including the GDrive token) before posting

15 
[GDrive]
type = drive
scope = drive
token = xxx
team_drive = xxx
server_side_across_configs = true

[FTP]
type = sftp
host = xxx
user = xxx
port = xxx
pass = *** ENCRYPTED ***
md5sum_command = none
sha1sum_command = none```
16

Thanks, so your FTP is an SFTP server?

Can you tell a bit more about it? (for me to evaluate performance)

17

yes it is SFTP, i tried FTP connect but for unknown reason I could not able to connect with rclone. later I tried SFTP which is working fine.

18

OK, is it your own SFTP server or some storage service? which?

19

it's just a server, there is some site is also running.

20

Why are you so secretive? it makes it impossible to understand your situation and then we cannot help you.

What do you know about the server? that must be more than "just a server"