What is the problem you are having with rclone?
I am trying to upload Confluence backups from Atlassian Cloud to a Google Storage Bucket.
So far, I have successfully uploaded files using a configuration file.
Now, we need to optimize this and make it more dynamic in the sense that we don't want to use config files. Instead, we need to be able to pass the config parameters directly in the command with connections strings.
To begin with, I am trying to download the confluence backup to my local machine and I am failing with passing the headers in the connection string. Can you please advise if it's possible to pass HTTP headers with connection strings? If yes, how?
Run the command 'rclone version' and share the full output of the command.
rclone v1.62.2
- os/version: Microsoft Windows 10 Pro 21H2 (64 bit)
- os/kernel: 10.0.19044.2965 Build 19044.2965.2965 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: cmount
Which cloud storage system are you using? (eg Google Drive)
Google Cloud Storage
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone copy ":http,url='https://<my_cloudsite>.atlassian.net/wiki/download/temp/filestore/':'ea99194c-5199-4568-bb84-6f7c177d889c',headers='Content-Type,application/json,Authorization,<my_authorization_token>'" C:\TEMP
The rclone config contents with secrets removed.
The following config file works fine
[gcs]
type = google cloud storage
object_acl = private
bucket_acl = private
location = europe-north1
storage_class = NEARLINE
env_auth = false
token = <my_token>
project_number = <my_gcp_project>
[atlas-conf]
type = http
url = https://<my_cloudsite>.atlassian.net/wiki/download/temp/filestore/
headers = Content-Type,application/json,Authorization,,<my_authorization_token>
A log from the command with the -vv
flag
This response is from Atlassian Cloud. I think it doesn't receive the authorization header.
2023/05/22 15:50:39 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "copy" ":http,url='https://<my_cloudsite>.atlassian.net/wiki/download/temp/filestore/':'ea99194c-5199-4568-bb84-6f7c177d889c',headers='Content-Type,application/json,Authorization,<mytoken>'" "'C:\\TEMP'" "-vv"]
2023/05/22 15:50:39 DEBUG : Creating backend with remote ":http,url='https://<my_cloudsite>.atlassian.net/wiki/download/temp/filestore/':'ea99194c-5199-4568-bb84-6f7c177d889c',headers='Content-Type,application/json,Authorization,<mytoken>'"
2023/05/22 15:50:39 DEBUG : Using config file from "C:\\Users\\dimit\\AppData\\Roaming\\rclone\\rclone.conf"
2023/05/22 15:50:39 DEBUG : :http: detected overridden config - adding "{vOGmu}" suffix to name
2023/05/22 15:50:39 DEBUG : Assuming path is a file as HEAD response is an error (403 Forbidden)
2023/05/22 15:50:39 DEBUG : If path is a directory you must add a trailing '/'
2023/05/22 15:50:39 DEBUG : Root: https://<my_cloudsite>.atlassian.net/wiki/download/temp/filestore/%27ea99194c-5199-4568-bb84-6f7c177d889c%27,headers=%27Content-Type,application/
2023/05/22 15:50:39 DEBUG : fs cache: adding new entry for parent of ":http,url='https://<my_cloudsite>.atlassian.net/wiki/download/temp/filestore/':'ea99194c-5199-4568-bb84-6f7c177d889c',headers='Content-Type,application/json,Authorization,<>'", ":http{vOGmu}:'ea99194c-5199-4568-bb84-6f7c177d889c',headers='Content-Type,application/json,Authorization,<>'"
2023/05/22 15:50:39 DEBUG : Creating backend with remote "'C:\\TEMP'"
2023/05/22 15:50:39 Failed to create file system for "'C:\\TEMP'": config name contains invalid characters - may only contain numbers, letters, `_`, `-`, `.`, `+`, `@` and space, while not start with `-` or space, and not end with space