I've been using rclone for a while, and I've finally got the energy to try and automate things a bit more (I've got 400+GB in Azure blob storage as archive - cost me £0.72 last month!).
What is the problem you are having with rclone?
I can run rclone sync on my Ubuntu server to Azure from the command line, but I can't get my bash script to work (throwing a "error reading destination directory" error in the logs)
What is your rclone version (output from rclone version
)
Rclone version: 1.51.0
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Running ubuntu 18:04 (all packages updated)
Which cloud storage system are you using? (eg Google Drive)
Using Azure blob storage (with no issues until this point). For what it's worth, I'm setting files to 'ARCHIVE' tier once they are uploaded, as this is last resort backups.
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
This is the command I run directly from the command line (which works):
root@gomez:~/bin# rclone sync /zpool/Media/Media/Pictures azure:pictures -v
2020/07/27 22:29:53 INFO : Azure container pictures: Waiting for checks to finish
2020/07/27 22:29:53 INFO : Azure container pictures: Waiting for transfers to finish
2020/07/27 22:29:53 INFO : Waiting for deletions to finish
2020/07/27 22:29:53 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks: 51788 / 51788, 100%
Elapsed time: 0.0s
This is the contents of the script:
#!/bin/bash
/usr/bin/rclone sync /zpool/Media/Media/Pictures azure:pictures: --log-file /var/log/rclone-sync-pictures-sync.log --log-level DEBUG
now="$(date +"%d/%m/%y")"
mail -s "Cron - Weekly Azure Backup ${now}" redacted@gmail.com < /var/log/rclone-sync-pictures-sync.log
The rclone config contents with secrets removed.
[azure]
type = azureblob
account = gomezbackup
key = REDACTED
A log from the command with the -vv
flag
And this is the output of the mentioned log file:
2020/07/27 22:55:29 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "sync" "/zpool/Media/Media/Pictures" "azure:pictures:" "--log-file" "/var/log/rclone-sync-pictures-sync.log" "-$
2020/07/27 22:55:29 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2020/07/27 22:55:29 ERROR : : error reading destination directory: -> github.com/rclone/rclone/vendor/github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /home/runner/work/rclone/src/github$
===== RESPONSE ERROR (ServiceCode=InvalidResourceName) =====
Description=The specifed resource name contains invalid characters.
RequestId:2e90c81f-101e-0053-6360-64861b000000
Time:2020-07-27T21:55:29.7275193Z, Details:
Code: InvalidResourceName
GET https://gomezbackup.blob.core.windows.net/pictures:?comp=list&delimiter=%2F&include=metadata&maxresults=5000&restype=container&timeout=31536001
Authorization: REDACTED
User-Agent: [rclone/v1.51.0]
X-Ms-Client-Request-Id: [9556bf12-6d4c-4c88-62e5-be036806ce4a]
X-Ms-Date: [Mon, 27 Jul 2020 21:55:29 GMT]
X-Ms-Version: [2018-11-09]
--------------------------------------------------------------------------------
RESPONSE Status: 400 The specifed resource name contains invalid characters.
Content-Length: [243]
Content-Type: [application/xml]
Date: [Mon, 27 Jul 2020 21:55:28 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Error-Code: [InvalidResourceName]
X-Ms-Request-Id: [2e90c81f-101e-0053-6360-64861b000000]
X-Ms-Version: [2018-11-09]
2020/07/27 22:55:29 INFO : Azure container pictures:: Waiting for checks to finish
2020/07/27 22:55:29 INFO : Azure container pictures:: Waiting for transfers to finish
2020/07/27 22:55:29 ERROR : Azure container pictures:: not deleting files as there were IO errors
2020/07/27 22:55:29 ERROR : Azure container pictures:: not deleting directories as there were IO errors
2020/07/27 22:55:29 ERROR : Attempt 1/3 failed with 1 errors and: -> github.com/rclone/rclone/vendor/github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /home/runner/work/rclone/src/github.$
===== RESPONSE ERROR (ServiceCode=InvalidResourceName) =====
Description=The specifed resource name contains invalid characters.
RequestId:2e90c81f-101e-0053-6360-64861b000000
Time:2020-07-27T21:55:29.7275193Z, Details:
Code: InvalidResourceName
GET https://gomezbackup.blob.core.windows.net/pictures:?comp=list&delimiter=%2F&include=metadata&maxresults=5000&restype=container&timeout=31536001
Authorization: REDACTED
User-Agent: [rclone/v1.51.0]
X-Ms-Client-Request-Id: [9556bf12-6d4c-4c88-62e5-be036806ce4a]
X-Ms-Date: [Mon, 27 Jul 2020 21:55:29 GMT]
X-Ms-Version: [2018-11-09]
--------------------------------------------------------------------------------
RESPONSE Status: 400 The specifed resource name contains invalid characters.
Content-Length: [243]
Content-Type: [application/xml]
Date: [Mon, 27 Jul 2020 21:55:28 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Error-Code: [InvalidResourceName]
X-Ms-Request-Id: [2e90c81f-101e-0053-6360-64861b000000]
X-Ms-Version: [2018-11-09]