[SOLVED] Crash with (beta) rclone mount after a few days... log included

Help and Support
1

Hello there (and sorry for my english)

OS : windows 10 x64
rclone version : rclone 1.52.3gb6d3cad7-beta , but I've this problem since a few versions already. Can't find the real culprit : /

Command line used : C:\rclone-browser-1.2-a1156a0-win64\rclone.exe mount --cache-dir t:\mountcache --attr-timeout 8700h --stats 10s --buffer-size 16M --vfs-cache-poll-interval 30s --poll-interval 30s --dir-cache-time 8700h --vfs-cache-max-age 24h --async-read=true --vfs-read-wait 20ms --vfs-write-wait 20s --vfs-cache-mode writes --vfs-read-chunk-size 16M --vfs-read-chunk-size-limit 1G --use-mmap --local-no-check-updated --drive-chunk-size=64M --multi-thread-streams=4 --multi-thread-cutoff 250M --transfers 8 --drive-disable-http2=true --fast-list --vfs-case-insensitive --vfs-cache-max-size 15G --log-level ERROR --log-file="c:\rclonelog\rclonelog.txt" --rc "gdriveencryptedownapi:seedold" G:

(I know some parameters have no effect)

Problem : Rclone mount crash and die after a few days. Pretty light usage. I'm seeding from the mount, but I limit the speed, I don't use full cache, nothing hardcore... My computer is not stressed when the crash happens....... And it's only since a few versions.

Two crashes in the log, one a few days ago, and one a few minutes ago :

https://web.rootax.org/temp/rclonelog.zip

I hope it can help fix this bug, if I'm not the only one impacted ?

Thx : )

2

wow, never seen so many flags ever.

i would suggest that you know what each flag does and if is needed.
for example,

  • --fast-list does nothing on a mount.

  • --transfers does nothing on a mount.

  • --drive-disable-http2=true, the default is true.

  • --use-mmap - never found a reason to use that flag.
    as per docs It is possible this does not work well on all platforms so it is disabled by default

  • if you are seeding/reading from the mount then why use
    --vfs-cache-mode writes

  • what version of winfsp?

3

--fast-list is not used when I "warm" the cache directory with rclone.exe -vvv rc vfs/refresh recursive=true after ?

--use-mmap , i find that the memory is released faster after a transfer stoped, so I keep that.

And I'm using cache mode writes, because I read at one point that some function /files operation were not supported with minimal ?

For the rest and some others, yeah i know it not needed/used, but it's easier to keep them and change them when I'm doing some testing.

I'm using WinFsp 2020.2 Beta2.

4

  • --fast-list does nothing on a mount so you can remove it.
    but add it to rc vfs/refresh recursive=true --fast-list

  • --use-mmap
    given the documentation does not work well on all platforms usually means windows os.
    and if you read the logs, there seems to be memory errors
    runtime.sigpanic() runtime.memmove(0x23600620000, 0x23606150000, 0x1000)
    i suggest to remove that flag and test.

  • how much ram do you have?

  • how much ram does rclone and winfsp use?

5

Noted for fast-list, thx.

About nmap, I'll try without it. But I used it for months until this point, without problems.

I've 32gb of ram. I never paid attention about winfsp memory usage, and rclone varies between about 100mo to 500mo, depending on the the number of transferts at the same time. In any case my ram is far from full.

6

so you have been running that exact mount command month after month after month without crashing and now all of a sudden, it crashes every few days?

  • beta = bugs
    never used the beta of rclone
    never used the beta of winfsp
    never had a mount crash

  • if you are going to use the beta of rclone for mounts then make sure you have read this
    Testing for new vfs cache mode features

7

Well, that's why I posted this with the log, so it can help finding what's wrong... I'm not complaining or anything... And I participated in the new vfs cache mode topic, I know the drill...

8

sure, just trying to helpful not overbearing.

we all learn the vfs cache drill :upside_down_face:

might be this https://github.com/golang/go/issues/37470

9

Like @asdffdsa said, start with a much simpler mount. I've been using the latest beta of both rclone and WinFsp for close to four years now. Never had a mount crash. I'm not saying it can't happen, but it very likely has something to do with some of your flags.

Oh, and --fast-list doesn't do anything for the refresh command either.

I don't have a use case for cache or writing to the mount, so mine is as follows:

rclone mount --attr-timeout 1000h --dir-cache-time 1000h --poll-interval 0 --rc --read-only -v

My prime command:

rclone rc vfs/refresh recursive=true --timeout 10m

My writable mount, when I need it:

rclone mount -v

Hope that helps a fellow Windows user :slight_smile:

1 Like
10

Well, my mindset is, the crashes are pretty new, and I've used theses flags for a long times without a problem. If a flag / settings is causing a problem now, it's still usefull to report it and try to fix it, if the problem is located

11

It's definitely useful to report crashes without a doubt, I think everyone agrees with that. Thanks for reporting it!

On a separate note though, removing none useful flags just makes things cleaner. I only add a flag if I have a reason to add it and otherwise, I use defaults as my 'simple' approach to rclone and pretty much everything.

12

when did you install the beta of winfsp, perhaps that is problem?

13

The day of the release, so 22 days ago. I don't believe it's linked when I'm look at what changed vs the non beta, but I'll revert to stable if I've crashes again (right now i'm just testing without nmap, i prefer testing one thing at a time).

14

great,

be sure to update us

15

So, yeah, it's seems that it was the -nmap option caused the random crash. I've a lot of ram, so it's not a big deal disabling it.

16

Hmm, we should probably try to fix that. Was there a full backtrace in the zip above? On mobile so can't check myself at the moment.

17

Sorry I don't know what you mean by full backtrace. It was just the output of rclone started with the --log-level ERROR command.

18

Sadly on Windows, I have no idea where you'd find that as depending on how you started it, that is what would capture it as on Linux, it would be systemd.

Do you see the stack trace in the log you created?

19

I'm looking for something which looks like this

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0xe3ae60]

goroutine 20886 [running]:
github.com/rclone/rclone/backend/putio.(*Object).Open(0xc000185e40, 0x1d4eee0, 0xc000d7af00, 0xc0008fedf0, 0x1, 0x1, 0x0, 0x1, 0x80, 0x80)
	github.com/rclone/rclone/backend/putio/object.go:251 +0x220
github.com/rclone/rclone/fs/operations.(*ReOpen).open(0xc001209500, 0xc001209500, 0xc000a35ca0)
	github.com/rclone/rclone/fs/operations/reopen.go:96 +0x349
github.com/rclone/rclone/fs/operations.NewReOpen(0x1d4eee0, 0xc000d7af00, 0x1d64f00, 0xc000185e40, 0xa, 0xc0008fede0, 0x1, 0x1, 0x0, 0x0, ...)
	github.com/rclone/rclone/fs/operations/reopen.go:47 +0x10c
github.com/rclone/rclone/fs/operations.(*multiThreadCopyState).copyStream(0xc0005402d0, 0x1d4eee0, 0xc000d7af00, 0x0, 0x0, 0x0)
	github.com/rclone/rclone/fs/operations/multithread.go:74 +0x326
github.com/rclone/rclone/fs/operations.multiThreadCopy.func1(0xc0000ec700, 0x0)
	github.com/rclone/rclone/fs/operations/multithread.go:173 +0x45
github.com/rclone/rclone/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1(0xc0007595c0, 0xc0007596e0)
	github.com/rclone/rclone/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
created by github.com/rclone/rclone/vendor/golang.org/x/sync/errgroup.(*Group).Go
	github.com/rclone/rclone/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66
20

There is a lot of goroutine message in the log I posted, but no panic : runtime error.