What is the problem you are having with rclone?
I have a management server controlling the replication of data between 3 servers via the use of RCD. The issue im facing is that it is ignoring the "checkFirst": true variable. I've confirmed that the normal rclone sync method using the same variables work, just not with this method.
Run the command 'rclone version' and share the full output of the command.
rclone v1.69.1
- os/version: debian 12.9 (64 bit)
- os/kernel: 6.1.0-30-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.24.0
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
NA/ Local Storage (ext4/SFTP)
The command you were trying to run
Management Server issuing command to server two too replicate to server three
curl -X POST "http://10.0.8.102:5572/sync/sync" \
-H "Content-Type: application/json" \
-d '{
"srcFs": "/mnt/data",
"dstFs": "NAS03-10GB:/mnt/data",
"fastList": true,
"checkFirst": true,
"transfers": 64,
"checkers": 128,
"sizeOnly": true,
"ignoreChecksum": true,
"update": true,
"bufferSize": "512M",
"useMmap": true,
"maxTransfer": "100G",
"connectTimeout": "20s",
"timeout": "20s",
"retries": 1,
"metadata": true,
"noUpdateDirModTime": true,
"noUpdateModTime": true,
"bufferSize": "256M",
"multiThreadStreams": 4,
"multiThreadCutoff": "64M",
"cutoffMode": "hard",
"orderBy": "size,desc",
"tpsLimit": 0,
"tpsLimitBurst": 0,
"bwLimit": "off",
"disableFeatures": "ping,timestamp",
"maxBacklog": 200000,
"sftpSetModTime": false,
"sftpDisableConcurrentWrites": true,
"sftpDisableConcurrentReads": true,
"sftpUseFstat": false,
"sftpDisableHashCheck": true,
"sftpChunkSize": "32M",
"sftpConcurrency": 16,
"sftpSetModTime": false,
"sftpCompression": false,
"filters": {
"excludeRule": [
"@Rec*/**",
".streams/**",
"**.log",
"Download/**",
"SHARE/**"
]
},
"logFile": "datacopylogfile-all.log",
"logLevel": "INFO",
"_async": true
}'
rclone running on each server using the below command:
./rclone rcd --rc-no-auth --rc-addr :5572 --rc-allow-origin="*" --rc-enable-metrics --log-level=DEBUG --log-file=./rclone-debug.log
output from watch -n 1 'curl -X POST http://10.0.8.103:5572/core/stats'
Every 1.0s: curl -X POST http://10.0.8.102:5572/core/stats proxmox-mgmt-01: Tue May 13 14:12:53 2025
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 2272 0 2272 0 0 706k 0 --:--:-- --:--:-- --:--:-- 739k
{
"bytes": 20224233368,
"checking": [
"ROMs/gba/Dr. Graga DrawPixel by Graga (PD).gba",
"ROMs/gba/Dr. Graga Interupt Example. (PD).gba",
"ROMs/gba/Dr. Graga Pira-Pira's Birthday Present (PD).gba",
"ROMs/gba/Dr. Graga brpsgift (PD).gba",
"ROMs/gba/Dr. Graga carcar (PD).gba",
"ROMs/gba/Dr. Graga carfun (PD).gba",
"ROMs/gba/Dr. Graga demo0 (PD).gba",
"ROMs/gba/Dr. Graga demo1 (PD).gba"
],
"checks": 23003,
"deletedDirs": 0,
"deletes": 0,
"elapsedTime": 8.277796631,
"errors": 0,
"eta": 6784,
"fatalError": false,
"renames": 0,
"retryError": false,
"serverSideCopies": 0,
"serverSideCopyBytes": 0,
"serverSideMoveBytes": 0,
"serverSideMoves": 0,
"speed": 145123905.55086657,
"totalBytes": 1004857755593,
"totalChecks": 33019,
"totalTransfers": 23,
"transferTime": 99.133439472,
"transferring": [
{
"bytes": 6587543552,
"dstFs": "NAS03-10GB:/mnt/data",
"eta": 2152,
"group": "job/3",
"name": "Archive/Games/PC/Action/God of War Ragnarok-RUNE/God.of.War.Ragnarok-RUNE/rune-god.of.war.ragnarok.iso",
"percentage": 5,
"size": 121660866560,
"speed": 66649910.278734624,
"speedAvg": 53453668.80313769,
"srcFs": "/mnt/data"
},
{
"bytes": 6165196800,
"dstFs": "NAS03-10GB:/mnt/data",
"eta": 188,
"group": "job/3",
"name": "Archive/Games/PS3/Gran_Turismo_6_USA_PS3-CLANDESTiNE/BCUS98296/PS3_GAME/USRDIR/GT.VOL",
"percentage": 40,
"size": 15407310848,
"speed": 62393417.054390736,
"speedAvg": 48912450.45504717,
"srcFs": "/mnt/data"
},
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[NAS01-10GB]
type = sftp
host = XXX
pass = XXX
shell_type = unix
use_insecure_cipher = true
disable_hashcheck = true
key_file = /root/.ssh/id_rsa
[NAS02-10GB]
type = sftp
host = XXX
pass = XXX
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum
key_file = /root/.ssh/id_rsa
[NAS03-10GB]
type = sftp
host = XXX
pass = XXX
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum
key_file = /root/.ssh/id_rsa
A log from the command that you were trying to run with the -vv flag
https://privatebin.net/?6826568cc0cf10d2#B3XtqqM11cEqNPa5zWKD4tutLjwfQe1FQYqjE5if9K7d