Rclone mount/mounts crashing/disconnecting from the mount folder/point (panic: runtime error: invalid memory address or nil pointer dereference)

What is the problem you are having with rclone?

am having this issue where my mount will disocnnected/ crash. I beleive it happens when there is a lot of io going on becuase that is the only way i can replicated. i have tried all possible combinations to see if i can pin poin this issue but i have failed. at this point am not sure if this is related to my os, kernel, rclone itself or somehting else.

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.4

  • os/version: debian 12.4 (64 bit)
  • os/kernel: 6.7.3-zabbly+ (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.22rc2
  • go/linking: static
  • go/tags: none
    aniel@Hz:~$

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
-->

Which cloud storage system are you using? (eg Google Drive)

using dropbox, google, ftp one and few others

The command you were trying to run (eg rclone copy /tmp remote:tmp)

[Unit]
Description=rclone_mounts
#Wants=network-online.target
#After=network-online.target
 
[Service]
Type=notify
User=aniel
Group=aniel
Environment=RCLONE_CONFIG=/home/aniel/.config/rclone/rclone.conf
ExecStart=/usr/bin/rclone mount mounts: /media/mergerfs/mounts \
	#--config=/home/aniel/.config/rclone/rclone.conf \
	--log-file=/media/logs/mounts.log \
	--user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/122.0" \
	#--vfs-write-back 10m \
	#--min-age 1s \
	--ignore-existing \
	#--vfs-fast-fingerprint \
	#--rc-no-auth \
	#--rc-addr 127.0.0.1:5575 \
	#--rc \
	--allow-other \
	#--allow-non-empty \
	--log-level=NOTICE \
	--dir-cache-time 9999h \
	#--vfs-cache-poll-interval 1s \
	--vfs-cache-max-age 9999h \
	#--vfs-refresh=true \
	--vfs-read-chunk-size 1M \
	--vfs-read-chunk-size-limit off \
	#--cache-dir=/media/mergerfs/local_hdd/rclone_cache \
	--vfs-cache-mode full \
	--vfs-cache-max-size 100G \
	#--buffer-size 0M \
	#--vfs-read-ahead 32M \
	--tpslimit 10 \
	--tpslimit-burst 10 \
	#--multi-thread-streams 0 \
	#--cutoff-mode cautious \
	#--network-mode \
	#--read-only \
	#--order-by size,mixed,75 \
	#--max-backlog 10000 \
	--transfers 32 \
	--checkers 64 \
	#--contimeout 30m \
	#--timeout 30m \
	#--retries-sleep 60s \
	--umask 002
ExecStop=/usr/bin/fusermount -uz /media/mergerfs/mounts
#ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --url 127.0.0.1:5575 _async=true
Restart=always
RestartSec=10

[Install]
WantedBy=network-online.target

also tried this one and even simpler ones plus diorecly mounting it on the terminal

rclone mount mounts: /media/mergerfs/mounts --log-file=/media/logs/mounts.log --vfs-cache-mode full --vfs-cache-max-size 100G --umask 002 --ignore-existing --allow-other
[Unit]
Description=rclone_mounts
Wants=network-online.target
After=network-online.target
 
[Service]
User=aniel
Group=aniel
Type=simple

ExecStart=/usr/bin/rclone mount mounts: /media/mergerfs/mounts --log-file=/media/logs/mounts.log --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/122.0" --ignore-existing --allow-other --log-level=NOTICE --dir-cache-time 9999h --vfs-cache-max-age 9999h --vfs-read-chunk-size 1M --vfs-read-chunk-size-limit off --vfs-cache-mode full --vfs-cache-max-size 100G --tpslimit 10 --tpslimit-burst 10 --transfers 32 --checkers 64 --umask 002


# Perform lazy unmount
ExecStop=/usr/bin/fusermount -zu /media/mergerfs/mounts

# Restart the service whenever rclone exists with non-zero exit code
Restart=on-failure
RestartSec=15

[Install]
# Autostart after reboot
WantedBy=default.target

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.


[aniel_personal_gdrive]
type = drive
scope = drive
token = XXX
team_drive =
root_folder_id =

[dave_ftp]
type = ftp
host = XXX
user = XXX
pass = XXX
port = 21

[dave_ftp_chunker]
type = chunker
remote = dave_ftp:
chunk_size = 5Gi

[dave_ftp_encrypted]
type = crypt
remote = dave_ftp_chunker:xxx
filename_encoding = base32768
password = XXX

[dropbox]
type = dropbox
token = XXX

[dropbox_encrypted]
type = crypt
remote = dropbox:xxx
filename_encoding = base32768
password = XXX
password2 = XXX

[media_merged]
type = union
action_policy = all
create_policy = all
search_policy = ff
upstreams = "xxx_encrypted:" "dropbox_encrypted::ro"

[mounts]
type = combine
upstreams = "media_merged=media_merged:" "dropbox=dropbox:" "dropbox_encrypted=dropbox_encrypted:" "aniel_personal_gdrive=aniel_personal_gdrive:" "xxx=xxx:" "xxx_encrypted=xxx_encrypted:" "dave_ftp=dave_ftp:" "dave_ftp_encrypted=dave_ftp_encrypted:"

[xxx]
type = xxx
access_token = XXX
api_host = XXX
encrypt_files = false

[xxx_encrypted]
type = crypt
remote = xxx:xxx
filename_encoding = base32768
password = XXX
chunk_size = 1Gi

A log from the command that you were trying to run with the -vv flag

this is the log right before the crash for my understanding

2024/02/06 10:18:19 NOTICE: union root '': srcU.f=union root '', srcURemote="media/to fix/shows/1600.Penn.S01E02.The.Skiplantic.Ocean.1080p.AMZN.WEB-DL.DD+5.1.x264-AJP69", dstURemote="media/to fix/shows/_UNPACK_1600.Penn.S01E02.The.Skiplantic.Ocean.1080p.AMZN.WEB-DL.DD+5.1.x264-AJP69.1"
2024/02/06 10:18:25 NOTICE: union root '': srcU.f=union root '', srcURemote="media/to fix/shows/_UNPACK_1600.Penn.S01E02.The.Skiplantic.Ocean.1080p.AMZN.WEB-DL.DD+5.1.x264-AJP69.1", dstURemote="media/to fix/shows/1600.Penn.S01E02.The.Skiplantic.Ocean.1080p.AMZN.WEB-DL.DD+5.1.x264-AJP69.1"
2024/02/06 10:19:03 NOTICE: union root '': srcU.f=union root '', srcURemote="media/to fix/shows/1600.Penn.S01E03.So.You.Dont.Want.to.Dance.1080p.AMZN.WEB-DL.DD+5.1.x264-AJP69", dstURemote="media/to fix/shows/_UNPACK_1600.Penn.S01E03.So.You.Dont.Want.to.Dance.1080p.AMZN.WEB-DL.DD+5.1.x264-AJP69"
2024/02/06 10:19:41 NOTICE: union root '': srcU.f=union root '', srcURemote="media/to fix/shows/_UNPACK_1600.Penn.S01E03.So.You.Dont.Want.to.Dance.1080p.AMZN.WEB-DL.DD+5.1.x264-AJP69", dstURemote="media/to fix/shows/1600.Penn.S01E03.So.You.Dont.Want.to.Dance.1080p.AMZN.WEB-DL.DD+5.1.x264-AJP69"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x195f2a9]

goroutine 79501 [running]:
github.com/rclone/rclone/backend/xxx.(*Fs).findObject(0xc00612e000, {0x2ae3248, 0xc015e5a780}, {0xc011f82600, 0xf5}, {0xc0112b87d5, 0x4d})
	github.com/rclone/rclone/backend/xxx/xxx.go:376 +0x429
github.com/rclone/rclone/backend/xxx.(*Fs).NewObject(0xc00612e000, {0x2ae3248, 0xc015e5a780}, {0xc016160c80, 0x13d})
	github.com/rclone/rclone/backend/xxx/xxx.go:461 +0x59
github.com/rclone/rclone/backend/crypt.(*Fs).NewObject(0xc000540240, {0x2ae3248, 0xc015e5a780}, {0xc015e5522d?, 0xc006108401?})
	github.com/rclone/rclone/backend/crypt/crypt.go:419 +0x8b
github.com/rclone/rclone/backend/union.(*Fs).NewObject.func1(0x0)
	github.com/rclone/rclone/backend/union/union.go:713 +0x8b
github.com/rclone/rclone/backend/union.multithread.func1()
	github.com/rclone/rclone/backend/union/union.go:977 +0x54
created by github.com/rclone/rclone/backend/union.multithread in goroutine 79497
	github.com/rclone/rclone/backend/union/union.go:975 +0x47

i have research this but havent found a fix yet. here are some of those links

thank you in advance.

It is some custom built rclone:) As for now the latest official rclone release is v1.65.2 and beta v1.66.

You have to seek answers with whoever created it.

When running such experimental system (your Linux kernel is also far ahead what Debian supports officially) you have to deal with issues yourself I am afraid.... or within community doing it. I guess there are some reasons for using such on the edge system.

i can go back to official and stable kernel. do u think that will make a difference? does it even makes a difference in this case. and yes i did forgot to mention that am running acustom rclone build. do u think i should test with the latest rclone beta build?

What I am saying is that if you are running such experimental/custom rclone/OS nobody can even try to replicate your issues. So you have to fix it yourself.

Unless there are other people on this forum who share your setup and can help.

ok i will try to replicate the issue using official kernel and latest rclone beta build and report back

Let's see - if it helps with your endeavour into unknown even better.

ahha fair enought.

maybe this is the problem - some let's say 3rd party remote.

Because this is what you use in union and mount

I did not tried taking this out of the equation. however i did took ot out of the main equation by eliminating it from the union and the issue stil happened. do u think it might be causing the issue just by being in the combine group?

I have no clue. You are using something custom built with some "secret" remote.

But genuinely I think you should contact creator of xxx remote to discuss your issues. Or hope that somebody reading this forum is using xxx too and has more knowledge what is doing.

you have a good point and i will do that, thank for replying to me.

Good luck. And push them to submit PR with this new remote. Might be useful for others too.

yeah i dont understand why he dosent push this to the official rclone branch. i mean i found his project from this forum so there is no secrecy there. am just the way am and have my reason as to why not reveal said name/ project.

Fair enough. Let's see where it goes

1 Like

can confirm swithcing to the latest rclone beta fixes the issue. i will report this to him/her and hopefully it can be fixed and added back to the latest rclone code branch.

1 Like