I’ve got an HTTP server that’s behing a user//password combination in order to access it, and I’d like to use rclone. I can’t figure out how to actually get rclone to register that I’m passing it a username//password though.
ash-4.3# rclone lsd remote: -vv
2019/01/25 23:55:26 DEBUG : rclone: Version “v1.45” starting with parameters [“rclone” “lsd” “remote:” “-vv”]
2019/01/25 23:55:26 DEBUG : Using config file from “/root/.config/rclone/rclone.conf”
2019/01/25 23:55:27 ERROR : : error listing: error listing “”: failed to readDir: HTTP Error 401: 401 Unauthorized
2019/01/25 23:55:27 Failed to lsd: error listing “”: failed to readDir: HTTP Error 401: 401 Unauthorized
and the config is
[remote]
type = http
url = https://insert_url_here
username = encrypted
password = encrypted
pass = encrypted
user = encrypted
I added those last 4 fields manually using the
rclone config password field key
but they don’t seem to work. Is there any way to do this?
I’ve seen that with other things, like wget, but it’s not quite working here.
ash-4.3# rclone sync remote:/ /volume1/location
2019/02/03 23:39:55 Failed to create file system for “remote:/”: parse “https://user:password@site”/: first path segment in URL cannot contain colon
Sorry to bother again but it isn’t quite working still.
ash-4.3# rclone copy remote: . --retries 10
2019/02/04 12:10:01 ERROR : E: Failed to copy: failed to open source object: Open failed: HTTP Error 404: 404 Not Found
2019/02/04 12:10:01 ERROR : I: Failed to copy: failed to open source object: Open failed: HTTP Error 404: 404 Not Found
2019/02/04 12:10:01 ERROR : M: Failed to copy: failed to open source object: Open failed: HTTP Error 404: 404 Not Found
2019/02/04 12:10:01 ERROR : Oneshots: Failed to copy: failed to open source object: Open failed: HTTP Error 404: 404 Not Found
2019/02/04 12:10:03 ERROR : Attempt 1/10 failed with 4 errors and: failed to open source object: Open failed: HTTP Error 404: 404 Not Found
2019/02/04 12:10:10 ERROR : E: Failed to copy: failed to open source object: Open failed: HTTP Error 404: 404 Not Found
2019/02/04 12:10:10 ERROR : Attempt 2/10 failed with 1 errors and: failed to open source object: Open failed: HTTP Error 404: 404 Not Found
2019/02/04 12:10:17 ERROR : Attempt 3/10 succeeded
It “succeeds” but the end result is a local folder full of HTML files, and it doesn’t crawl through the folders recursively, like how wget would. I think it should be doing this automatically, no? Or is there something that I should be setting.
I don’t think that I can. There’s no way for me to change my password on the site and I can’t create a new account for you to mess with, so I"m not really comfortable sharing the access details. Is there anything else I could provide that could help with diagnosis, logs or anything?
Try running with rclone lsf -vv remote: - does that print the expected top level files and directories? If not then rclone isn’t parsing the HTML page properly.
ash-4.3# rclone lsf -vv remote:
2019/02/05 12:16:36 DEBUG : rclone: Version “v1.45” starting with parameters [“rclone” “lsf” “-vv” “remote:”]
2019/02/05 12:16:36 DEBUG : Using config file from “/root/.config/rclone/rclone.conf”
A
B
C
D
E
F
G
H
I
J
K
L
M
N
Non-English
O
P
Q
R
README.txt
S
Subfolders.txt
T
U
V
W
X
Y
Z
Some other files
2019/02/05 12:16:45 DEBUG : 4 go routines active
2019/02/05 12:16:45 DEBUG : rclone: Version “v1.45” finishing with parameters [“rclone” “lsf” “-vv” “remote:”]
All of the directories are there, as well as the files, just the same that I see it in a normal web browser.
Yes it looks like rclone is confused by the links to the directories which appear without a / on the end in the HTML.
If you go to one of the sub pages in your browser, say the A directory, does it show a / on the end in the URL bar? I’m wondering if the site redirects to something with a / on the end - this would be something rclone could detect easily.
If that doesn’t work then I could put a flag in to treat all text/html pages as directories.
