What is the problem you are having with rclone?
I'm running rclone with the command line from below. If I copy a large single file (around 1GB) into the mounted directory this goes really fast as expected (because of the cache). According to the webgui this file is uploaded then and this upload finishes a few minutes after the copy ended. So everything fine here. What I noticed is that the copy command doesn't get back after that (I also tried to copy with pv file.img > /jottacloud-folder/file.img to see a status bar, it was really fast but at 100% it stopped and did not terminate) . I'm not sure if the reason for that is rclone or something else. But the file is copied correctly
But if I copy 50 files, about 3MB each, about 150MB overall they don't seem to hit the cache. The process of copying seemingly takes exactly as long as the upload takes (and the upload speed limit seems to be ignored). If I look into the cache folder I can see that the files only appear there one after another really slow. Looking at the webgui it almost looks like they hit the cache folder at the same time as they're uploaded.
The setup is the same in both scenarios. If I copy the files from one folder on the hard drive to another (both not rclone mounted) it goes really fast so my filesystem/SSD isn't the bottleneck here.
Is it possible that after copying the file into the cache rclone does not tell the program which issued the copy that it's finished now (=in cache) but only after it's uploaded?
What is your rclone version (output from rclone version)
rclone v1.52.3
- os/arch: linux/arm
- go version: go1.14.7
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Raspberry Pi OS, 32bit on a Raspberry Pi 4 (4GB Ram) booting from SSD attached via USB 3.0
Which cloud storage system are you using? (eg Google Drive)
Jottacloud
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone mount jottacloud:/backups /home/pi/mnt/jottacloud/backups --retries 5 --retries-sleep 5m --cache-dir ~/rcloneCache --vfs-cache-mode full --vfs-cache-max-size 350G -vv --log-file log.txt --rc --rc-web-gui --rc-htpasswd ./htpasswd --rc-serve --rc-web-gui-no-open-browser --bwlimit "04:00,off 07:00,600K 16:00,350K" --rc-addr :5572 --umask 770 --allow-other
Note: The cache directory lies on the (fast) SSD (not SD card). There is more than 400GB of free space on this SD so rclone could use the whole 350GB of cache that I allow it to use.
A log from the command with the -vv flag
Log was a bit too big for pastebin, so here as file:
Note: Don't get confused by the directory jottacloud. This is not a mounted folder, only backups inside of this is the mounted jottacloud. The non-cloud folder is the one with the data that I copy into the backups folder (which is the mounted jottacloud)
rclone-log.txt (499.5 KB)
Thanks in advance
