Rclone stopped working

Help and Support
1

my problem is that rclone stopped working to serve content to my Plex server. I can see the drive letter and even browse content (only content that existed as of the morning when the problem started) but I can't open it, download or upload via rclone.

I also run filestream in parallel with rclone to upload my files. This issue started shortly after the filestream instance (two different drive letters) choked on a file that wouldn't sync and I had to force it closed then do a hard reboot. The rclone instance was running via NSSM and I had it set up as per that old, infamous Reddit thread. Clunky as it was it worked fine since last year some time. I even updated rclone at least once during that time without issue. If I upload files via Filestream, they do sync properly and appear even if I login and look at the GDrive portal directly so I know the data is getting there. It's like something is stuck and the drive created by rclone is not talking to GDrive anymore. The content in my rclone drive letter is not matching my Filestream drive when I add new content to the cloud after the incident this morning.

In the process of troubleshooting I have renamed the old mount and created a new one in rclone using the original name as to not mess up the other config. I have turned off the NSSM method and created a native startup item in Windows using task manager. The task will start and run. It creates and I can see the drive letter as well as browse. However, I can't upload, download or view content from that rclone-created drive. Again, it still works fine in filestream.

I am frustrated at this point. I don't share my server outside my home, but my ADD 10 year that is out of school until late August is climbing the walls because he can't watch most of his stuff as it's stored on GDrive. My understanding is that you do NOT want to use Filestream as your actual server entry point because of API usage or something to that effect so I haven't even tried that. I need to get rclone working again.

What is your rclone version (output from rclone version)

rclone v1.51.0

  • os/arch: windows/amd64
  • go version: go1.13.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows10 64bit.

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

mount --allow-other --dir-cache-time 72h --drive-chunk-size 64M --log-level INFO --vfs-read-chunk-size 32M --vfs-read-chunk-size-limit off gdrive: Z: --config "C:\Users\user\.config\rclone\rclone.conf" --vfs-cache-mode writes

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

Still working on this.....

2

hi,
--allow-other does nothing on windows, you can remove it.

what version of winfsp are you using, latest stable is v1.6.

i can understand the need to get that 10 year old watch videos, asap.
what kind of device does the 10yr use to watch video?
where are you running plex, on a local server, computer, cellphone or tablet?
android, ios or what?

you might want to read this
https://rclone.org/commands/rclone_mount/#directory-cache
"changes done on the remote will only be picked up once the cache expires."

3

Hi @asdffdsa. Thanks for the help and sorry for the slow response. I have had this page open since 6AM and I have been pecking away at it since then between real work! I literally had to change the greeting from "Good morning" to "Hi" because it took so long!

This morning my system was sort of working on it's own again with no further input on my part. I have no idea what the issue was yesterday. I looked at my GDrive usage thinking perhaps I had exceeded my API limit, but it didn't even appear close to that and I wasn't prevented from accessing the files using Filestream so I don't think that was it. I don't share my server so virtually all of my traffic is in my own house meaning I have a good idea of how it is used. Nothing has changed in recent days or weeks or even months.

I removed the --allow-other part of the command line. Not sure what it does, but it was part of the original instructions I had used to set this all up and hasn't seemed to cause an issue. I couldn't find the command/flag listed in either of these places:

https://rclone.org/commands/
https://rclone.org/flags/

I am running the latest WinFSP. I was at 1.5.X before yesterday and in the process of troubleshooting I installed the latest v1.6.20027. It didn't make a difference as of when I went to bed last night so I don't know it's what "fixed" this issue, but I do like staying up to date on software.

The devices I use are 98% dedicated streaming players. Shield, Roku, Firestick etc. Some I consume on desktop via PMP, but mostly streaming devices.

I am running Plex server on a Win10-64 box that is on 24/7. It's a Core-i5 and I've never had an issue with too much load whether local or pulling content from GDrive. Most of my devices will direct play so transcoding is also rarely a factor.

That directory cache page is interesting. I modified my command line to this:

mount --dir-cache-time 1m --drive-chunk-size 64M --log-level DEBUG --vfs-read-chunk-size 32M --vfs-read-chunk-size-limit off gdrive: Z: --config "C:\Users\flkar\.config\rclone\rclone.conf" --vfs-cache-mode writes --log-file=C:\rclone\rclonemount.txt

I didn't see anything in the docs about specific units or limits on this parameter. I changed it from "72h" to "1m". It is working so far and seems to update quickly! Hopefully it's not an issue! Thanks for the hint!

More generally, I have disabled the NSSM method that was in use as recommended in the original Reddit thread I used to set all this up initially. I am now using the Windows Task Scheduler and created a task that would start rclone with the above command line whenever the system starts. I tried to run it as my own user, but when I do that rclone does not create a drive letter. I had to set it up as the SYSTEM user. Do you recommend this method? Is there an entirely better method to do this with?

Thanks again for your input. I know you jumped in on another thread I had a few months ago so I appreciate your experience!

4

so to be clear, everything is working for the 10yr?

nssm is good, but i use task scheduler, mounting as system user.

about --dir-cache-time 72h, gdrive has limits in terms of api calls and setting that value to 1m, might cause spikes of api calls.

you might want to read this, it is long but a lot of good info in there.

also, you might want to check out this command rclone serve dlna remote:
this would turn rclone into a dlna server, and any video player, streaming stick would see it as a 'instant' plex server.
i actually run rclone on my cellphone using rclone serve.

5

It has limits but that's not why must people on gdrive set that high. It's because gdrive has a change tracking mechanism that notifies rclone to drop the cache for that particular folder/file(s) when things change. So there's not a good reason to keep it lower.

6

good to know, thanks

7

So this reared it's ugly head again. Possibly a different issue, but I hadn't changed anything on my end.

Last night I was watching a movie that was hosted on GDrive. It just stopped, period about 20 minutes in. Rebooting the server...twice, the Firecube or even trying the movie/file on my desktop PC in PMP didn't allow it to play. I'll get the dreaded "not powerful enough". I even tried it in VLC directly on my server that rclone is running on and it says "your input can't be opened".

However, Plex, VLC anything else I try to run works perfectly fine using local files so it's something to do with my mount. Again, I hadn't changed anything in weeks. It has been running OK using task scheduler. Not as reliable as NSSM, but simpler too. An occasional task restart or Plex server restart and it would take off again.

So this morning I tried to update rclone from 1.52 to 1.52.1 as well as the latest version of WinFSP. I was one version behind on that.

Rebooted. Still won't work. The drive letter mounts and I can see the files, but I can play them, even on that local machine use a standalone video player lik eVLC, MPV or Windows Movies etc. It's like the mount has a TOC, but no actual access to the files themselves.

Anybody seen this before or have any ideas? My command line is this:

mount --rc --dir-cache-time 72h --drive-chunk-size 64M --log-level DEBUG --vfs-read-chunk-size 32M --vfs-read-chunk-size-limit off gdrive: Z: --config "C:\Users\flkar\.config\rclone\rclone.conf" --vfs-cache-mode writes --log-file=C:\rclone\rclonemount.txt

The only mod I have made to this since the problem started was to change my cache time back to 72hr when it was 1hr. It had been running like that for weeks without issue though so I don't know what happened last night to cause this problem.

I also just noticed I still have the logging level set to "DEBUG". I just changed that to "NOTICE" but the drive still doesn't work.

If I look at my Google Console the only thing that seems odd is my errors by drive.files.get:

image

I will admit that this showing the initial spike about 9PM last night is about the exact time it just "stopped" working for seemingly no reason. I'm not going to pretend to understand exactly what that means, but I didn't come close to hitting my quotas.

Any idea what can cause these errors? is it something wrong with my command line for example?

Appreciate the help!

9

[quote="asdffdsa, post:8, topic:16313"]
i am confused, you wrote that you already updated to the latest stable, which is v1.6.20027.
so which version of winfsp are you using, some kind of beta?
not good to use beta software when there is no need for it.[/quote]
No sir. I didn't say what version I was running. However, I went from v1.7.20123 to v1.7.20172. I got them from here:

Best I can tell that is the stable. It was released 21 hours ago.

Not sure what you mean by that. The only thing I set up is what was in that command line above. It does have several cache-related parameters in it. Is that what you are referring to?

10

not vfs cache flags, but the cache backend remote, which reddit loves for some reason.

when streaming media, i always mount as --read-only

are you still running two mounts at the same time?

11

have you excluded the z: from anti-virus scanner, including the built-in windows.

12

I have no idea. If it was something that I would have had to set up it would be in the command line wouldn't it? AFAIK, that is the only place I have ever configured parameters unless there was something in the NSSM setup that I'm not using anymore and don't recall.

Can I simply add the "--read-only" flag to the command line to make it read only? That's easy stuff. lol

Not sure what you mean by this. I am running one instance of rclone and one instance of Drive Fielstream. The Filestream mount works flawlessly. I can even play the files that won't play in Plex from the rclone mount.

My practice is that I usually upload new files to the drive via Filestream, but only play them via the rclone mount. My drive letter is mapped that way for the purposes of Plex. I do occasionally upload something through my rclone mount drive and did in fact do that last night with a couple files, but didn't know that/if it is an issue to do that if Filestream is running.

The only reason I ever upload that way is because if I look at the drive capacity via Filestream, it shows 10TB total. if I look at the drive mounted by rclone it shows 1PB. I don't know if it's true, but it's acting like I'm "filling up" the drive when using Filestream as it shows it about half full now even though my actual space is unlimited..

I have not manually done that, but it's a good point! I use Windows Defender and frankly, I've never tried, therefore, don't know how to exclude drives. I will look at that now!

13

Update, I just excluded both the Filestream drive and the rclone mount drive letters from my virus scans.

14

about that spike,
did filestream do that?
or
did rclone do that?

15

How would I know that for sure? I am looking at the stats and I filtered it JUST for the OAuth client which I presume is my rclone instance. That is where the spike still shows up.

16

i do not use gdrive much but you can create multiple client ids.

17

Is that version of WinFSP the current one as it appeared to be? Is that first one actually a beta? It had blue badge on it and wasn't labeled beta.

18

about the version of winfsp, i see that the latest stable is now 1.7, and that happened a few hours ago.
so sorry about that confusion.

why is that a problem?
that is a quirk of rclone

are you having a problem uploading using rclone mount?

19

Not a complaint. I'm not concerned about the 1PT, rclone limit. I'm worried about the 10TB Filestream limit! lol

Don't know if there are chronic issues. I don't usually use it for uploading because Filstream just seems to be more responsive and I have a little icon that tells me the progress of any uploads.....:wink:

I never considered a correlation before, but last night I did try rclone mount (as well as Filestream) to upload and it hosed. Could be a complete coincidence. Don't know.

20

we are spending a lot of time because you want to use two different tools, at the same time, to mount your gdrive.
and the reason for not using rclone for uploads is lack of a progress indicator?

you are using --rc, should you should be able to see what is going on, using a web browser.
or use

21

i do not believe in coincidences