I have a Windows 7 computer that I am trying to use with Radarr, Sonarr, Jackett, Deluge, and Rclone mount to load files to Gdrive. Once Deluge completely downloads a few files and pushes the completed files to the Rclone mount it locks up the the mount/file explorer. At this point I can no longer navigate to the mount in Windows file explorer, and Radarr (what I’m primary using) also produces a system error that the drive is unavailable. If I disconnect and reconnect the mount it is immediately fine and I start back at square one.
I was using the latest released version of Rclone but saw another post which I cannot seem to locate now that advised using the latest beta - I tried that with the same results.
I am mounting with the following command:
rclone mount --allow-other --vfs-cache-mode writes gcrypt: Q:
Having the same issue as OP - copying from local drive to mounted gdrive (Im using FreeFileSync to do so). Just tried this beta version and it seems to be a little better. The transfer still seems to lock up, but eventually it gets itself sorted and moves to the next file. The problem is it seems to do this on every file, sometimes failing the copy and having to retry.
This is the post I was looking for - and I started running the beta listed in that post. It did not fix the issue for me. I just loaded the new beta and will try that.
Unlike ModuRaziel my issue doesn't seem to resolve itself, I have to stop rclone and remount which fixes the issue temporarily.
I will post my -vv log from last night (as soon as I figure out how to host it for linking since it's huge) which involved multiple files being uploaded to Gdrive, as well as opening the mount in Windows file explorer. It looks like I had a few successful uploads before it locked up between 5AM and 9AM this morning.
The most useful thing of all for debugging would be to run the mount with --rc then use curl http://localhost:5572/debug/pprof/goroutine?debug=1 on it when it has locked up as outlined here. That will tell me whether it is a deadlock or not and if it is, where it is.
If it is a deadlock that will be enough info to fix it ().
Looking at those I don't see a deadlock which is good news.
However what appears to be happening is that in each case there are files being uploaded.
What that means is that
the calling app calls close()
this notifies the kernel which notifies rclone
rclone calls Flush()
rclone does not return until the file is uploaded
which means that the application is blocked while the file is uploading
This is what the patch was supposed to solve, but it obviously doesn't work
Perhaps what rclone needs is a more asynchronous upload, so instead of uploading it directly, rclone could stick it into a queue to be uploaded later.
Pros
rclone could limit the number of uploads
rclone could cancel uploads for files which were modified again
rclone could retry uploads
rclone could persist the queue so uploads would resume when rclone was restarted
Cons
it would make the upload a write-back operation, so the calling application would think the file was uploaded when it wasn't. Provided you only access the file through the mount everything will be consistent as rclone will use its local copy, but if you looked on the remote you'd see the file wasn't there yet.
I'm not the best person to answer that question but from my point of view that con is acceptable since the issue renders the mount virtually unusable in my case.
It's interesting I have 2 computers using an identical config file and mount commands, the one is just running the Plex server. The other that is having the problem is running Sonarr, Radarr, Jackett.
Im not sure how the con would affect my use-case, as I am just trying to upload files to gdrive as a backup of my local media. I wont even be accessing files on the gdrive unless there is some catastrophic event that completely wipes both my local and local backup copies.
WinFsp spawns a number of threads to service file system requests. This number if usually the number of cores you have on the machine with a minimum of 2. So if you have 1 or 2 cores you get 2 threads. If you have 8 cores you get 8 threads.
The FUSE layer of WinFsp has also an option ThreadCount which may be useful to increase the number of threads servicing file system requests. I am not certain if rclone exposes this option to the command line, but if it does try -o ThreadCount=16 to get 16 file system threads. This should be enough to service most workloads, but in extreme cases you can also try -o ThreadCount=64.
If it doesn't take you more than a minute, could you please warm up that vfs-async branch for version 1.48? (hopefully this is the best way of asking...) Thanks very much!
).