iDrive - mounting for streaming

Help and Support
1

What is the problem you are having with rclone?

I've been using iDrive E2 for some time, and it worked well in the beginning, but then upload speed started being quiet slow (up to 10MB/s), and now even download speed is slow (about the same speed). It is connected to a server with a 1gps connection. Use case is with mounting and VFS, but even a simple copy is slow with default settings.

Run the command 'rclone version' and share the full output of the command.

rclone v1.67.0

  • os/version: ubuntu 22.04 (64 bit)
  • os/kernel: 6.2.0-060200-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.22.4
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

iDrive E2

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy iDrive:test . -v --progress

or

[Unit]
Description=RClone Mount Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/dan/.config/rclone/rclone.conf
#KillMode=none
RestartSec=5
ExecStart=/usr/bin/rclone mount iDrive: /home/dan/mnt/idrive \
# This is for allowing users other than the user running rclone access to the mount
--allow-other \
# Google Drive is a polling remote so this value can be set very high and any changes are detected via polling.
--dir-cache-time 720h \
# The log level output
--log-level NOTICE \
# This is setting the file permission on the mount to user and group have the same access and other can read
--umask 002 \
# Please set this to your own value below
--user-agent danelem \
# This is used for caching files to local disk for streaming
--vfs-cache-mode full \
# This limits the cache size to the value below
--vfs-cache-max-size 10G \
# This limits the age in the cache if the size is reached and it removes the oldest files first
--vfs-cache-max-age 6h \
#--vfs-read-chunk-streams 50 \
--s3-upload-concurrency 50
ExecStop=/bin/fusermount -u /home/dan/mnt/idrive
Restart=on-failure
User=dan
Group=media

[Install]
WantedBy=multi-user.target

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[iDrive]
type = s3
provider = IDrive
access_key_id = XXX
secret_access_key = XXX
endpoint = i0g0.fra.idrivee2-22.com

A log from the command that you were trying to run with the -vv flag

Not sure what log would help here.

I can see that playing with these flags help for upload:
--transfers 150
--s3-upload-concurrency 150
--s3-chunk-size 50M

And this for downloads:
--multi-thread-streams 50

But it doesn't work when using VFS cache, I think.

I don't know if someone has a config to share that works well, and if changing these settings is supposed to change that much. I also contacted Idrive support.

Thank you for sharing and for your help!

2

welcome to the forum,

need to use a debug log, --log-level DEBUG

such large values are often counter productive.

3

Right, actually even lower values like 20 seem to help, but I'm not sure it is really the more efficient way to do it.

4

I'm doing some extra tests, and it seems that downloading from a crypted remote is slower. Much slower than a non-crypted remote. Would this make sense?

5

not trying to be mean, but, to be honest, really hard to follow you posts.
you post about streaming, downloading, uploading, mount, copy, crypt.
then changing flags and values, using crazy high values, not posting exact command, debug logs, config.

fwiw, just have to do proper testing and compare the results, then you will know.
pick one issue a time, do all the testing on that and then move on to the next issue.
good luck.

6

Maybe I wasn't clear.

I am trying to isolate the problem that I don't think is connected to rclone. I am more interested to know if the same issue happened to others.

I've been with iDrive for about a year for multiple projects. Recently all the speeds have dropped. Both download and upload. Since it worked fine for months, and I didn't change anything, it seems that external factors are responsible for it.

Notice that it could be my provider too.

7

I tried iDrive for a few months for streaming but it didn't work for me in the end.

I started with Germany location, being closest to my server. It started fast, but then speeds dropped to the point where streaming was not possible any more.

I opened another bucket in Paris, moved everything over. It worked well for a few days, then it didn't.

Ultimately, speeds were very inconsistent, and frequently dropped below what I'd need for my streaming, so I concluded iDrive is not suitable.

I tried various mount settings, nothing worked. It is definitely an iDrive problem.

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.