I've been using rclone to sync from Dreamhost's DreamObjects service for quite some time successfully. Lately though I'm getting poor performance and strange errors. I'm sure this is on the Dreamhost side but maybe there are workarounds I can do on the rclone side.
I was thinking of trying to turn off concurrent downloads, but I don't see an option for that in the S3 flags.
What is your rclone version (output from rclone version)
Which OS you are using and how many bits (eg Windows 7, 64 bit)
not sure this is a problem with dreamhost, as you are getting errors about spare files?
might try to disable that --local-no-sparse Disable sparse files for multi-thread downloads
could be an issue with rclone
what is file system type fo the network drive, ntfs or what?
if you sync to local storage, such as the c: drive, do you get errors?
Thanks! This is on the latest Windows 10 x64. The network drive presents as NTFS. It's a Drobo NAS.
I think the sparse errors have been showing for a while, but I've ignored them and it's been ok. I added --local-no-sparse to my command. Those errors went away.
I was searching the flags for "concurrent" so I missed the --transfers=1 option. I added that too.
I'm still getting errors though. The transfer works for a while and then dies. See attached.
I received a response from Dreamhost, as this seemed to start after an incident the other day.
Our engineers have diagnosed and resolved this issue, however the long term fix is still in the process of being applied. As such, the effects of the outage may take a few days to fully return back to normal, however uptime should improve over time.
So maybe I just need to wait a while and try again.
I have written to DreamHost support about the same issue several days ago. Since then I'm receiving daily email updates similar to what endquote wrote. In last email on Friday, they said they have resolved the issue and are preparing post-mortem analysis article.
I've now different issue with DreamObjects though and I'm not sure what is the cause. Rclone complains that it can't read metadata of data/0/504 object, but when I list the bucket, the object seems to be there:
$ rclone ls dreamobjects:my-backup/localhost/data/0/|grep 504
17 504
525185047 690
$ rclone ls dreamobjects:my-backup/localhost/data/0/504
$ rclone ls dreamobjects:my-backup/localhost/data/0/690
525185047 690
I have same exact backup on Backblaze B2 and there it seems I can read the metadata:
$ rclone ls b2:my-backup/localhost/data/0/|grep 504
17 504
525185047 690
$ rclone ls b2:my-backup/localhost/data/0/504
17 504
$ rclone ls b2:my-backup/localhost/data/0/690
525185047 690
Here's the error I'm getting when I try to sync my backup to Dreamobjects:
$ rclone sync -v /backup dreamobjects:my-backup/localhost
2021/06/28 11:06:32 NOTICE: data/0/504: Failed to read metadata: object not found
2021/06/28 11:06:32 ERROR : data/0/504: Failed to set modification time: object not found
2021/06/28 11:06:35 ERROR : S3 bucket my-backup path localhost: not deleting files as there were IO errors
2021/06/28 11:06:35 ERROR : S3 bucket my-backup path localhost: not deleting directories as there were IO errors
2021/06/28 11:06:35 ERROR : Attempt 1/3 failed with 2 errors and: not deleting files as there were IO errors
2021/06/28 11:06:41 NOTICE: data/0/504: Failed to read metadata: object not found
2021/06/28 11:06:41 ERROR : data/0/504: Failed to set modification time: object not found
2021/06/28 11:06:44 ERROR : S3 bucket my-backup path localhost: not deleting files as there were IO errors
2021/06/28 11:06:44 ERROR : S3 bucket my-backup path localhost: not deleting directories as there were IO errors
2021/06/28 11:06:44 ERROR : Attempt 2/3 failed with 2 errors and: not deleting files as there were IO errors
2021/06/28 11:06:49 NOTICE: data/0/504: Failed to read metadata: object not found
2021/06/28 11:06:50 ERROR : data/0/504: Failed to set modification time: object not found
2021/06/28 11:06:53 ERROR : S3 bucket my-backup path localhost: not deleting files as there were IO errors
2021/06/28 11:06:53 ERROR : S3 bucket my-backup path localhost: not deleting directories as there were IO errors
2021/06/28 11:06:53 ERROR : Attempt 3/3 failed with 2 errors and: not deleting files as there were IO errors
2021/06/28 11:06:53 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 2 (retrying may help)
Checks: 1620 / 1620, 100%
Elapsed time: 29.2s
2021/06/28 11:06:53 Failed to sync with 2 errors: last error was: not deleting files as there were IO errors
Hello, glad to join. Good to know, thank you for the info. I run the command, but unfortunately, I cannot include links in my post, so I spoofed it a little below to workaround the restriction.
Here's the output of the command you wrote
from B2
gist githubusercontent com /woky/4d861124235967055c55315a720a4396/raw/e70df36340d48782cb4294dcc7dff5b24a1ff514/rclone-b2.log
from Dreamobjects
gist githubusercontent com /woky/4d861124235967055c55315a720a4396/raw/e70df36340d48782cb4294dcc7dff5b24a1ff514/rclone-dreamobjects.log
i had asked for information, but you provided only some of it.
let's not waste posts going back and forth, with me trying to pull information from you, ok?
rclone checks for the existance of HEAD /woky-backup/bor/data/0/504
and dreamhost returns 404 Not Found
with dh, how did you copy the files into there, rclone, website or what?
also, no point in posting logs about b2, not going to help us.
with dh, how did you copy the files into there, rclone, website or what?
With rclone sync as in my first post today, but I run rclone sync several times during the time when DH had issues and was returning 503 (the topic of this thread). It's possible that some data were copied at this time. I have never used anything except rclone sync to push my data there.
