The successful setting for IDRIVE - Solution
Want to set rclone config for IDRIVE cloud
here it is:
type = s3
provider = Other
env_auth = false
access_key_id = XLS4OCF9Pxxxxxxxxxx
secret_access_key = dXjHOHFLP7OMPxxxxxxxxxxxxxxxxxxxxxx
region = us-east-1
acl = private
endpoint = s3 . us-west-1 . idrivecloud . io (remove space)
Set these and you will able to config IDRIVE
hello and welcome to the forum.
thanks for the info, did not know that idrive is offering a s3 clone. seems like a new offering.
i use wasabi, and idrive seems to be undercutting them.
i wonder what, if any, compromises, idrive is making.
seems that like wasabi, idrive, is about hot storage.
initial test, idrive is looking good.
rclone sync source dest --s3-chunk-size=100M
idrive
Transferred: 10G / 10 GBytes, 100%, 64.704 MBytes/s, ETA 0s
Transferred: 2 / 2, 100%
Elapsed time: 2m38.8s
wasabi
Transferred: 10G / 10 GBytes, 100%, 78.750 MBytes/s, ETA 0s
Transferred: 2 / 2, 100%
Elapsed time: 2m10.3s
if rclone is going to add idrive as a backend, here is one thing that will need to be dealt with
seems that the
--s3-max-upload-parts = 1000
2020/12/03 20:20:38 DEBUG : 5GB.01.file: multipart upload starting chunk 998 size 5M offset 4.868G/5G
2020/12/03 20:20:39 DEBUG : 5GB.01.file: multipart upload starting chunk 999 size 5M offset 4.873G/5G
2020/12/03 20:20:39 DEBUG : 5GB.01.file: multipart upload starting chunk 1000 size 5M offset 4.878G/5G
2020/12/03 20:20:39 DEBUG : 5GB.01.file: multipart upload starting chunk 1001 size 5M offset 4.883G/5G
2020/12/03 20:20:39 DEBUG : pacer: low level retry 1/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: txc43f351a56df44eea60bb-005fc98ee8, host id: txc43f351a56df44eea60bb-005fc98ee8)
2020/12/03 20:20:39 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2020/12/03 20:20:39 DEBUG : pacer: Reducing sleep to 0s
2020/12/03 20:20:39 DEBUG : 5GB.01.file: multipart upload starting chunk 1002 size 5M offset 4.888G/5G
2020/12/03 20:20:39 DEBUG : pacer: low level retry 1/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx511fe7de9ed7487886ed8-005fc98ee8, host id: tx511fe7de9ed7487886ed8-005fc98ee8)
2020/12/03 20:20:39 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2020/12/03 20:20:40 DEBUG : pacer: Reducing sleep to 0s
2020/12/03 20:20:40 DEBUG : 5GB.01.file: multipart upload starting chunk 1003 size 5M offset 4.893G/5G
2020/12/03 20:20:40 DEBUG : pacer: low level retry 1/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx4e0a0dea3596472d91e79-005fc98ee9, host id: tx4e0a0dea3596472d91e79-005fc98ee9)
2020/12/03 20:20:40 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2020/12/03 20:20:40 DEBUG : pacer: low level retry 2/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1001&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": EOF)
2020/12/03 20:20:40 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2020/12/03 20:20:41 DEBUG : pacer: low level retry 3/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx49c22b693fa54190a4ebc-005fc98eea, host id: tx49c22b693fa54190a4ebc-005fc98eea)
2020/12/03 20:20:41 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2020/12/03 20:20:41 DEBUG : pacer: low level retry 2/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1002&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": EOF)
2020/12/03 20:20:41 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2020/12/03 20:20:41 DEBUG : pacer: low level retry 2/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1003&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": EOF)
2020/12/03 20:20:41 DEBUG : pacer: Rate limited, increasing sleep to 160ms
2020/12/03 20:20:41 DEBUG : pacer: low level retry 3/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx4310361911884f0393636-005fc98eea, host id: tx4310361911884f0393636-005fc98eea)
2020/12/03 20:20:41 DEBUG : pacer: Rate limited, increasing sleep to 320ms
2020/12/03 20:20:41 DEBUG : pacer: Reducing sleep to 240ms
2020/12/03 20:20:41 DEBUG : 5GB.01.file: multipart upload starting chunk 1004 size 5M offset 4.897G/5G
2020/12/03 20:20:41 DEBUG : pacer: low level retry 4/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx863781fffcf34e738bbf7-005fc98eea, host id: tx863781fffcf34e738bbf7-005fc98eea)
2020/12/03 20:20:41 DEBUG : pacer: Rate limited, increasing sleep to 480ms
2020/12/03 20:20:42 DEBUG : pacer: low level retry 4/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1001&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59345->216.150.215.6:443: wsasend: An existing connection was forcibly closed by the remote host.)
2020/12/03 20:20:42 DEBUG : pacer: Rate limited, increasing sleep to 960ms
2020/12/03 20:20:42 DEBUG : pacer: low level retry 3/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1003&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59346->216.150.215.6:443: wsasend: An existing connection was forcibly closed by the remote host.)
2020/12/03 20:20:42 DEBUG : pacer: Rate limited, increasing sleep to 1.92s
2020/12/03 20:20:42 DEBUG : pacer: low level retry 5/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: txcf81cca9cd4c4ee2b7f49-005fc98eeb, host id: txcf81cca9cd4c4ee2b7f49-005fc98eeb)
2020/12/03 20:20:42 DEBUG : pacer: Rate limited, increasing sleep to 2s
2020/12/03 20:20:43 DEBUG : pacer: low level retry 1/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1004&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": EOF)
2020/12/03 20:20:44 DEBUG : pacer: low level retry 5/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1001&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59347->216.150.215.6:443: wsasend: An existing connection was forcibly closed by the remote host.)
2020/12/03 20:20:45 DEBUG : pacer: low level retry 4/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: txbd1e6ac03fea49f2b43b0-005fc98eee, host id: txbd1e6ac03fea49f2b43b0-005fc98eee)
2020/12/03 20:20:48 DEBUG : pacer: low level retry 6/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1002&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59348->216.150.215.6:443: wsasend: An existing connection was forcibly closed by the remote host.)
2020/12/03 20:20:49 DEBUG : pacer: low level retry 2/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: txe527f56ebb074b8693370-005fc98ef2, host id: txe527f56ebb074b8693370-005fc98ef2)
2020/12/03 20:20:52 DEBUG : pacer: low level retry 6/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1001&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59350->216.150.215.6:443: wsasend: An existing connection was forcibly closed by the remote host.)
2020/12/03 20:20:53 DEBUG : pacer: low level retry 5/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx6e9a3b1a6d25486fa4846-005fc98ef6, host id: tx6e9a3b1a6d25486fa4846-005fc98ef6)
2020/12/03 20:20:56 DEBUG : pacer: low level retry 7/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1002&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": set tcp 192.168.62.234:59351: use of closed network connection)
2020/12/03 20:20:57 DEBUG : pacer: low level retry 3/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx541dd662f6154531bf621-005fc98efa, host id: tx541dd662f6154531bf621-005fc98efa)
2020/12/03 20:21:00 DEBUG : pacer: low level retry 7/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1001&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59352->216.150.215.6:443: wsasend: An existing connection was forcibly closed by the remote host.)
2020/12/03 20:21:01 DEBUG : pacer: low level retry 6/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: txa55c7880fdcd40969900b-005fc98efe, host id: txa55c7880fdcd40969900b-005fc98efe)
2020/12/03 20:21:04 DEBUG : pacer: low level retry 8/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1002&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59353->216.150.215.6:443: use of closed network connection)
2020/12/03 20:21:05 DEBUG : pacer: low level retry 4/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx44632d18e4a9413fbaf75-005fc98f02, host id: tx44632d18e4a9413fbaf75-005fc98f02)
2020/12/03 20:21:08 DEBUG : pacer: low level retry 8/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1001&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59354->216.150.215.6:443: wsasend: An existing connection was forcibly closed by the remote host.)
2020/12/03 20:21:09 DEBUG : pacer: low level retry 7/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx9a4e214fcd754cd0b39c8-005fc98f06, host id: tx9a4e214fcd754cd0b39c8-005fc98f06)
2020/12/03 20:21:12 DEBUG : pacer: low level retry 9/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1002&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59355->216.150.215.6:443: wsasend: An existing connection was forcibly closed by the remote host.)
2020/12/03 20:21:13 DEBUG : pacer: low level retry 5/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx5058a711998841dc885d1-005fc98f0a, host id: tx5058a711998841dc885d1-005fc98f0a)
2020/12/03 20:21:16 DEBUG : pacer: low level retry 9/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1001&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59356->216.150.215.6:443: use of closed network connection)
2020/12/03 20:21:17 DEBUG : pacer: low level retry 8/10 (error InvalidArgument: Part number must be an integer between 1 and 1000, inclusive
status code: 400, request id: tx31fbcb0c0c57469185e66-005fc98f0e, host id: tx31fbcb0c0c57469185e66-005fc98f0e)
2020/12/03 20:21:20 DEBUG : pacer: low level retry 10/10 (error RequestError: send request failed
caused by: Put "https://s3.us-west-1.idrivecloud.io/asdffdsa01x5gb/5GB.01.file?partNumber=1002&uploadId=MmIwYTRiZmYtODYzNC00MDgyLTk1ZTItOTU1NzU4YWEzMGQx": write tcp 192.168.62.234:59357->216.150.215.6:443: use of closed network connection)
2020/12/03 20:21:21 DEBUG : pacer: low level retry 6/10 (error RequestCanceled: request context canceled
caused by: context canceled)
2020/12/03 20:21:23 DEBUG : pacer: low level retry 10/10 (error RequestCanceled: request context canceled
caused by: context canceled)
2020/12/03 20:21:25 DEBUG : pacer: low level retry 9/10 (error RequestCanceled: request context canceled
caused by: context canceled)
2020/12/03 20:21:27 DEBUG : pacer: low level retry 7/10 (error RequestCanceled: request context canceled
caused by: context canceled)
2020/12/03 20:21:29 DEBUG : pacer: low level retry 10/10 (error RequestCanceled: request context canceled
caused by: context canceled)
2020/12/03 20:21:31 DEBUG : pacer: low level retry 8/10 (error RequestCanceled: request context canceled
caused by: context canceled)
2020/12/03 20:21:33 DEBUG : pacer: low level retry 9/10 (error RequestCanceled: request context canceled
caused by: context canceled)
2020/12/03 20:21:35 DEBUG : pacer: low level retry 10/10 (error RequestCanceled: request context canceled
caused by: context canceled)
2020/12/03 20:21:35 DEBUG : 5GB.01.file: Cancelling multipart upload
2020/12/03 20:22:02 DEBUG : pacer: Reducing sleep to 1.5s
2020/12/03 20:22:05 DEBUG : pacer: Reducing sleep to 843.75ms
2020/12/03 20:22:05 DEBUG : 5GB.01.file: multipart upload starting chunk 5 size 5M offset 20M/5G
2020/12/03 20:22:05 DEBUG : pacer: Reducing sleep to 632.8125ms
2020/12/03 20:22:05 DEBUG : 5GB.01.file: multipart upload starting chunk 6 size 5M offset 25M/5G
2020/12/03 20:22:06 DEBUG : pacer: Reducing sleep to 474.609375ms
2020/12/03 20:22:06 DEBUG : 5GB.01.file: multipart upload starting chunk 7 size 5M offset 30M/5G
2020/12/03 20:22:07 DEBUG : pacer: Reducing sleep to 355.957031ms
``Thanks for writing
idrive prices are attractive
but wasabi is so expensive what i tested
Let us know @asdffdsa for 10 GB it took 2 minutes 38 sec, so fast, which two sources you have synced?
thanks a lot. will this work with free idrive account as well?
the source was 2 files of size 5GB for a total of 10GB
each file was random non-compressible bytes.
my internet is 1Gb/s fiber optic.
there does not seem to be a free account for idrive cloud, just a 30 day free trial
did not see a way to create service files, if true, i cannot use idrive.
another problem, they do not seem to offer payment using paypal
@asdffdsa great connection though mine is 250MB and also not having LAN connection to my computer. wifi modem is slow.
Yes you rightly said I asked idrive about the payment they don't have Paypal, leaving credit card numbers with hosting company is not desirable.
I simply need for backup, duplicati is good program. I used many cloud services, earlier digital ocean, then icedrive and now google workspace they offer for 3 months but its quite expensive. Idrive is good and economic at least one year one can try.
i use wasabi for hot storage only needs.
but for longer term backup i use aws s3.
i use combination of veeam software, fastcopy and 7zip to backup desktop and servers to a local backup server.
then using rclone, upload those files to aws s3 glacier for $4/TB/month
and using s3 life cycle policy, after 90 days, to move from s3 glacier to s3 glacier deep archive for 1$/TB/month
@asdffdsa thanks for sharing your experiences. With Duplicati I have automated folders which I need daily and some other weekly. I need not spend any time on these backups. Let me understand s3 glacier it seems to be not expensive.
the difference between veeam and duplicati.
duplicait is limited to backing up files.
veeam can backup files but also can create bare metal recovery images for disaster recovery.
so if a server, physical or virtual, failed to do a operating system update or ransomware, one click and i can role back the entire server to any previous backup.
or if the sever died or was stolen, i can use those recovery images to clone to an new hardware.
veeam also will backup sql servers and coordinate with any software that uses VSS, not simply backing up files using VSS.
Oh yes great program, this Duplicati is open source so far nothing wrong, but for professional use Veeam can be good and have ready support team. They have not mentioned prices so must be on higher side. Even idrive is providing server backup and economical too. 5 TB starting with 6 USD and from next year around 60$ which is not bad.
duplicati, do you use on micro$oft windows?
yes I am using on windows. Its open source program
i know this is off-topic but i wanted to use it, but two things made me nervous.
home page, for years, show that that the software is beta.
and here is chromium.
