Thanks for this @Animosity022
I have copied and pasted this, changing the variables to match my machine, but can't even get it to work locally (as per your example).
Will keep trying but I have no idea how this doesn't work. I've used rclone for years without issue.
Can you see any RC commands pass to the RCD at all?
I receive a job id but nothing happens. When calling stats it just shows all 0s.
Put the rcd in debug with -vv on the end and submit a job and then share the full debug log.
Here you go @Animosity022 - RCD on Docker, RC on Win10 (no error messages that I can see. Doesn't do anything)
RCD:
2020/08/19 18:41:50 DEBUG : rc: "sync/copy": with parameters map[dstFs:GoogleDrive_NAS_crypt:NAS srcFS:/data]
2020/08/19 18:41:50 DEBUG : rc: "sync/copy": reply map[jobid:138]:
2020/08/19 18:41:53 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 18:41:53 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
2020/08/19 18:41:53 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 18:41:53 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
2020/08/19 18:41:58 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 18:41:58 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
2020/08/19 18:41:58 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 18:41:58 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
And then on the controller:
C:\Users\James\Desktop\rClone\rclone-v1.46-windows-amd64>rclone rc sync/copy srcFS="/data" dstFs="GoogleDrive_NAS_crypt:NAS" --rc-addr 192.168.7.25:5572 _async=true --rc-user james --rc-pass rclone -vv
2020/08/19 19:41:50 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "rc" "sync/copy" "srcFS=/data" "dstFs=GoogleDrive_NAS_crypt:NAS" "--rc-addr" "192.168.7.25:5572" "_async=true" "--rc-user" "james" "--rc-pass" "rclone" "-vv"]
{
"jobid": 138
}
2020/08/19 19:41:50 DEBUG : 4 go routines active
If you aren't seeing any message when you type in the rclone rc command on the expected server, it's going somewhere else then. You'd have to troubleshoot your docker/network as it is going somewhere.
I find dockers for me add unneeded complexity and I don't use them for my home setup.
The rclone rc command is connecting to where you specified:
So for whatever reason, that doesn't seem to be the rclone rcd server/daemon you are watching.
But isn't the first line of my debug showing that the RCD received the copy command?
It even returns a job id that matches. Doesn't that mean that it was received?
We only have two things to really look at:
Rclone rcd - The server
Rclone rc - the client
If the client makes the request and the server receives it, we see a log.
If you are looking at the server and you don't see a log, it did not make it to that server, but somewhere else as you may have another rclone running if the job succeeded.
Just take the docker out of the mix, test with a simple few lines as I did and work your way back and see when it breaks down.
OK @Animosity022 - I am now running RC and RCD on the same Win10 machine.
When I run the command I receive no errors in the log, I receive a jobID, but nothing happens. (ignore the folder name v1.46 - I am running the latest rclone)
RC LOG
C:\Users\James\Desktop\rClone\rclone-v1.46-windows-amd64>rclone rc sync/copy srcFS="Z:" dstFs="GoogleDrive_NAS_crypt:NAS" _async=true --rc-user james --rc-pass rclone -vv
2020/08/19 21:09:52 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "rc" "sync/copy" "srcFS=Z:" "dstFs=GoogleDrive_NAS_crypt:NAS" "_async=true" "--rc-user" "james" "--rc-pass" "rclone" "-vv"]
{
"jobid": 13
}
2020/08/19 21:09:52 DEBUG : 4 go routines active
PS C:\Users\James\Desktop\rClone\rclone-v1.46-windows-amd64>
RCD LOG
2020/08/19 21:09:47 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
2020/08/19 21:09:52 DEBUG : rc: "sync/copy": with parameters map[dstFs:GoogleDrive_NAS_crypt:NAS srcFS:Z:]
2020/08/19 21:09:52 DEBUG : rc: "sync/copy": reply map[jobid:13]:
2020/08/19 21:09:52 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 21:09:52 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
2020/08/19 21:09:52 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 21:09:52 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
2020/08/19 21:09:57 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 21:09:57 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
Please remove that and share the output of the rc command.
OK some progress:
Failed to rc: Failed to read rc response: 401 Unauthorized: Unauthorized
Not sure why I am receiving that as I am able to explore remotes using the browser and upload using rclone non-remote with the same .conf
You'd have to compare the user / passwd from the RCD and the RC as one of them would be different.
_async jobs just go so it's hard to tell if you have syntax wrong unless you have a log file to check.
username and password are identical, copy and pasted the same file. I have pasted the logs that I receive. 401 is to do with GoogleDrive auth according to others. I have no idea why as I am able to browse my remote using that conf.
Perhaps it's a bug in the latest version of rclone with rc/rcd?
I don't think so as I'm running 1.52.3 and it's very well used with rc.
You'd have to be specific in including the full commands you are running along with the full debug output.
Does the command work without rc?
Yes, works fine. Has done for years.
So back to the original statement then as you'd have to share the full commands with full debug logs.
felix@guardian:~$ rclone rcd --rc-web-gui -vv --rc-addr 0.0.0.0:5572 --rc-user felix --rc-pass felix
2020/08/19 16:49:11 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "rcd" "--rc-web-gui" "-vv" "--rc-addr" "0.0.0.0:5572" "--rc-user" "felix" "--rc-pass" "felix"]
2020/08/19 16:49:11 NOTICE: Web GUI exists. Update skipped.
2020/08/19 16:49:11 NOTICE: Serving Web GUI
2020/08/19 16:49:11 INFO : Using --user felix --pass XXXX as authenticated user
2020/08/19 16:49:11 NOTICE: Serving remote control on http://[::]:5572/
2020/08/19 16:49:11 DEBUG : login_token "ZmVsaXg6ZmVsaXg="
2020/08/19 16:49:11 ERROR : Failed to open Web GUI in browser: exec: "xdg-open": executable file not found in $PATH. Manually access it at: http://felix:felix@[::]:5572/?login_token=ZmVsaXg6ZmVsaXg%3D
2020/08/19 16:49:23 INFO : /sync/copy: 127.0.0.1:53198: Unauthorized request from felix
and
felix@guardian:~$ rclone rc sync/copy srcFs="/home/felix/test" dstFs="GD:test" -vv --rc-user felix --rc-pass joe
2020/08/19 16:49:23 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "rc" "sync/copy" "srcFs=/home/felix/test" "dstFs=GD:test" "-vv" "--rc-user" "felix" "--rc-pass" "joe"]
2020/08/19 16:49:23 DEBUG : 4 go routines active
2020/08/19 16:49:23 Failed to rc: Failed to read rc response: 401 Unauthorized: Unauthorized
First I run the RCD:
rclone rcd --rc-web-gui -vv --rc-addr 0.0.0.0:5572 --rc-user james --rc-pass rclone
Then I run the RC:
rclone rc sync/copy srcFS="Z:\" dstFs="GoogleDrive_NAS_crypt:NAS" --rc-user james --rc-pass rclone -vv
RCD LOG
C:\Users\James\Desktop\rClone\rclone-v1.46-windows-amd64>rclone rcd --rc-web-gui -vv --rc-addr 0.0.0.0:5572 --rc-user james --rc-pass rclone
2020/08/19 21:48:39 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "rcd" "--rc-web-gui" "-vv" "--rc-addr" "0.0.0.0:5572" "--rc-user" "james" "--rc-pass" "rclone"]
2020/08/19 21:48:39 NOTICE: Web GUI exists. Update skipped.
2020/08/19 21:48:39 NOTICE: Serving Web GUI
2020/08/19 21:48:39 INFO : Using --user james --pass XXXX as authenticated user
2020/08/19 21:48:39 NOTICE: Serving remote control on http://[::]:5572/
2020/08/19 21:48:39 DEBUG : login_token "dmFtZFM6cmNsb32l"
2020/08/19 21:48:42 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 21:48:42 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
2020/08/19 21:48:42 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 21:48:42 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
2020/08/19 21:48:47 DEBUG : rc: "core/stats": with parameters map[]
2020/08/19 21:48:47 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletes:0 elapsedTime:0 errors:0 fatalError:false renames:0 retryError:false speed:0 transfers:0]:
RC LOG
PS C:\Users\James\Desktop\rclone docker> ./xrClone_RC.bat
C:\Users\James\Desktop\rclone docker>rclone rc sync/copy srcFS="Z:" dstFs="GoogleDrive_NAS_crypt:NAS" --rc-user james --rc-pass rclone -vv
2020/08/19 21:52:27 Failed to rc: Failed to read rc response: 401 Unauthorized: Unauthorized
PS C:\Users\James\Desktop\rclone docker>
That'd RC log never makes it to the RCD server as you don't get an authorized user so you are hitting something else.
Firewall? AV?
This line is missing:
2020/08/19 16:49:23 INFO : /sync/copy: 127.0.0.1:53198: Unauthorized request from felix
You get that if the RCD was telling you it's a bad password.
RC and RCD on same machine, firewall disabled. Configs as specified above with the correct credentials and syntax. I literally have no idea. Shall I post as a bug?
Update: Tried v1.52 - that doesn't work either.
Update 2: Managed to find a new debug message (I found that I was using dstFs - should be dstFS):
2020/08/19 23:39:31 DEBUG : rclone: Version "v1.52.1" starting with parameters ["C:\Users\James\Desktop\rclone-v1.52.1-windows-amd64\rclone.exe" "rc" "sync/copy" "srcFS=Z:" "dstFS=GoogleDrive_NAS_crypt:NAS" "--rc-user" "james" "--rc-pass" "rclone" "-vv"]
2020/08/19 23:39:31 DEBUG : 4 go routines active
2020/08/19 23:39:31 Failed to rc: Failed to read rc response: 400 Bad Request: {
"error": "Didn't find key "srcFs" in input",
"input": {
"dstFS": "GoogleDrive_NAS_crypt:NAS",
"srcFS": "Z:"
},
"path": "sync/copy",
"status": 400
}
PS C:\Users\James\Desktop\rclone-v1.52.1-windows-amd64>
@Animosity022 Fixed! It was my fault. I was misunderstanding how srcFs worked. I mounted a local then added the path afterwards and it's working.
The next issue is that my regular commands don't seem to be adhered to. I normally run with:
--checksum --transfers 1 --checkers 4 --contimeout 60s --timeout 300s --retries 3 --low-level-retries 10 --stats 1s --stats-file-name-length 0
but when I try this it uploads 4 files at once, and it is overwriting files that already exist (and shouldn't be with the checksum command). According to the global flags page I'm doing it correctly