Slow speeds downloading and uploading

Do you have a RAM disk that you could try setting the cache to? And copy the file to /dev/null?

Will help to figure out if the problem is with the disk or rclone.

Ok, so I've created a RAM disk and setup cache there, same results (even a bit worse) and write/read show much higher results.

- Test trying to copy from mount directly

sudo rsync --progress Movie.mkv /dev/null -P
Movie.mkv
2,503,611,382 100%    7.91MB/s    0:05:01 (xfr#1, to-chk=0/1)

Same file with rclone copy

Transferred:   	    2.332G / 2.332 GBytes, 100%, 32.423 MBytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:      1m20.9s

Write speeds on RAM disk

sudo dd if=/dev/zero of=/tmp/ramdisk/zero bs=4k count=100000
100000+0 records in
100000+0 records out
409600000 bytes (410 MB, 391 MiB) copied, 0.186891 s, 2.2 GB/s

Read speeds on RAM disk

sudo dd if=/tmp/ramdisk/zero of=/dev/null bs=4k count=100000
100000+0 records in
100000+0 records out
409600000 bytes (410 MB, 391 MiB) copied, 0.144923 s, 2.8 GB/s

Ok, as a catch-up, can you show the latest mount settings being used?

Also, reading online, it looks like ramdisk doesn't support sparse files, so you may need to create a block device from RAM and format it with ext4. Something like (adjust accordingly for your requirements):

mount -t tmpfs -o size=200M tmpfs temp/
cd temp/
dd if=/dev/zero of=disk.img bs=1M count=199
losetup /dev/loop0 disk.img
mkfs.ext4 /dev/loop0
mount /dev/loop0 temp2/

Also, paste the DEBUG logs again obtained from performing this test.

Ok, I did create the loop

tmpfs 10G 69M 10G 1% /temp
/dev/loop0 9.6G 37M 9.1G 1% /temp2

Then mounted on /temp2 and ran the same test to go back to the beginning, not going beyond 10/11M.

sudo rsync --progress /mnt/rclone/Movie.mkv /dev/null -P
Movie.mkv
2,503,611,382 100% 10.54MB/s 0:03:46 (xfr#1, to-chk=0/1)

The mount settings for this test and the last one are the same, just changing the cache location and size of the original mount I normally use.

rclone: Version "v1.53.1" starting with parameters;
--user-agent=cere
--allow-other
--dir-cache-time 1000h
--poll-interval 15s
--umask 002
-cache-dir=/temp2/ 
--vfs-cache-mode full 
--vfs-cache-max-size 10G
--log-level INFO
--log-file=/home/cere/rclonemount.log
--vfs-cache-max-age=336h
--config /home/cere/.config/rclone/rclone.conf
akira:UN /mnt/rclone/

I'd test without the mount all together and see your speeds.

Just run a rclone copy with a single file and download it and see if that gets you decent speeds as it may not be the mount or anything else other than you have bad peering to Google's API.

I think that experiment has already been done earlier in the thread:

Oops, I see that now. Thanks!

Yip, I have tested with rclone copy and it doesn't happen the issue is just with the mount.
I have even tried this docker but issue still persists :sleepy:

Can you do this test with rclone copy just to test like for like?

I did the test with the same file and rclone copy, is on the same comment. I don't have the issue there, see below;

Is that from the mount or remote? I believe @ncw is asking you test using rclone copy from the mount instead of directly from the remote.

Oh my bad I misunderstood @ncw request (that was from the remote)
Here is the output of doing both again

From Remote

rclone copy remote:Movie.mkv /home/cere/ -P
Transferred: 2.332G / 2.332 GBytes, 100%, 31.100 MBytes/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 1m24.3s

From Mount

rclone copy /mnt/rclone/Movie.mkv ~/descargas/ -P
Transferred: 2.332G / 2.332 GBytes, 100%, 10.464 MBytes/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 3m48.1s

Edit: I am currently on Kernel 5.7.0 and I have installed;
linux-image-4.19.0-10-amd64
linux-image-4.19.0-11-amd64
linux-image-5.7.0-0.bpo.2-amd64

I've tried changing my GRUB_DEFAULT from 0 to 1 and 2 (then update the grub) and reboot and no matter what it boots on 5.7.0

Thank you.

The problem is very clear there...

If you go to --vfs-cache-mode writes then the problem disappears - is that correct?

I'm assuming for all the tests you are doing this with an uncached file. I would expect if you do the test twice in a row that it should be much quicker as the second should use the cached version with --vfs-cache-mode full

If it isn't much quicker then that is pointing to your disks being slower than your network which is really odd!

So can you try the rclone copy from the mount twice in a row without clearing the cache?

Can you also try it with --multi-thread-streams 1 on the rclone copy command to see if that makes any difference?

It would be worth collecting IO statistics while it is running - you can run vmstat 5 in another terminal window for example

Thanks for your response @ncw

If you go to --vfs-cache-mode writes then the problem disappears - is that correct?

No, I talked to fast... when I did the test with the test I didn't remove the old file so it started at 100+MB/s but then went down :cry: (I've done it again with --vfs-cache-mode writes and is not going beyond 10/11MB/s)

I would expect if you do the test twice in a row that it should be much quicker as the second should use the cached version with --vfs-cache-mode full

  1. First time (uncached file) (IO Statistics)
rclone copy /mnt/rclone/Movie.mkv ~/ -P
Transferred:   	    2.332G / 2.332 GBytes, 100%, 10.440 MBytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:      3m55.3s
  1. Second time (file cached)(IO Statistics)
rclone copy /mnt/rclone/Movie.mkv ~/ -P 
Transferred:   	    2.332G / 2.332 GBytes, 100%, 364.509 MBytes/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:         6.6s

Can you also try it with --multi-thread-streams 1 on the rclone copy command to see if that makes any difference?

No difference when using flag --multi-thread-streams 1

If it isn't much quicker then that is pointing to your disks being slower than your network which is really odd!

If it was the disks I guess I could not download via web browser or torrents at solid 90-95MB/s or use rclone copy from a remote and get above 30MB/s for a single file no?
The only thing I can think of by reading other users problems is the Kernel. I've been able to go back to 4.19.0-11-amd64 but the issue still remains. Is there any specific Kernel for Debian I can try?

Correct.

Hmm if it is a kernel problem with FUSE then the Second time (file cached) test would be slow too. So I don't think it can be that.

If it was a kernel networking problem then the use rclone copy from a remote and get above 30MB/s for a single file would be slow, so it can't be that.

So what can it be???

Can you try your mount without the --bwlimit-file 30M flag? I know you said you tried that above, but I'd like to see the speed measurements if possible. That is the only flag which is the slightest bit unusual.

UPDATE: FYI @ncw , I have tested with plexdrive and speeds when accessing files from the mount are the same 10/11MB/s. So could it be FUSE? (I don't know if they share anything else apart from that)

Can you try your mount without the --bwlimit-file 30M flag? I know you said you tried that above, but I'd like to see the speed measurements if possible. That is the only flag which is the slightest bit unusual.

Same file uncached from mount without --bwlimit-file flag

rclone copy Movie.mkv ~/ -P
Transferred: 2.332G / 2.332 GBytes, 100%, 7.692 MBytes/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 5m10.4s

 eno1  /  traffic statistics

                           rx         |       tx
--------------------------------------+------------------
  bytes                     2.45 GiB  |       21.01 MiB
--------------------------------------+------------------
          max          101.41 Mbit/s  |      873 kbit/s
      average           65.65 Mbit/s  |   550.73 kbit/s
          min              60 kbit/s  |        6 kbit/s
--------------------------------------+------------------
  packets                    1802883  |          297669
--------------------------------------+------------------
          max               8656 p/s  |        1476 p/s
      average               5634 p/s  |         930 p/s
          min                111 p/s  |           6 p/s
--------------------------------------+------------------
  time                  5.33 minutes

Same file but from remote

rclone copy -P "akira:Movie.mkv" ~/
Transferred: 2.332G / 2.332 GBytes, 100%, 27.859 MBytes/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 1m34.0s

 eno1  /  traffic statistics

                           rx         |       tx
--------------------------------------+------------------
  bytes                     2.45 GiB  |       25.05 MiB
--------------------------------------+------------------
          max          350.54 Mbit/s  |     3.89 Mbit/s
      average          219.32 Mbit/s  |     2.19 Mbit/s
          min              50 kbit/s  |       16 kbit/s
--------------------------------------+------------------
  packets                    1780063  |          370426
--------------------------------------+------------------
          max              29534 p/s  |        6923 p/s
      average              18542 p/s  |        3858 p/s
          min                 91 p/s  |          11 p/s
--------------------------------------+------------------
  time                  1.60 minutes

With rclone v1.50.2 and --vfs-cache-mode full the limitation of 10MB/s disappears (but the issue is to have to wait until the file is fully downloaded to open) :woozy_face:

rclone copy Movie.mkv ~/ -P
Transferred:   	    2.332G / 2.332 GBytes, 100%, 27.202 MBytes/s, ETA 0s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            1 / 1, 100%
Elapsed time:     1m27.7s


                           rx         |       tx
--------------------------------------+------------------
  bytes                     1.56 GiB  |       14.72 MiB
--------------------------------------+------------------
          max          396.54 Mbit/s  |     4.18 Mbit/s
      average          191.51 Mbit/s  |     1.76 Mbit/s
          min              20 kbit/s  |        1 kbit/s
--------------------------------------+------------------
  packets                    1132738  |          217436

It could be...

Rclone had some problems with the kernel fuse implementation recently. I thought it was all fixed up...

You could try adding --async-read=false to your rclone mount - this will replicate the 1.50.2 FUSE system better.

it seems like v1.50.2 don't like --async-read=false flag

2020/10/06 21:55:34 Fatal error: unknown flag: --async-read

Use that for 1.53 and test as that parameter didn't exist in the lower versions.