cberni
January 21, 2022, 2:53am
1
rclone mount is using a lot of RAM memory and doesn't seem to free up.
I would have a server with 88gb ram and the mount is using 22gb after 24h working. With a service restart the mount comes back to almost 0gb.
The remote backent have a lot of small files (millions).
Is there a flag to limit the rclone mount memory usage?
rclone v1.57.0
os/version: ubuntu 20.04 (64 bit)
os/kernel: 5.4.0-94-generic (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.17.2
go/linking: static
go/tags: none
google cloud storage
rclone copy /tmp remote:tmp)
rclone mount remote:bucket /remote --umask=0002 --allow-other --default-permissions
[remote]
type = google cloud storage
service_account_file =
RCLONE_VFS_CACHE_MODE=full
RCLONE_VFS_CACHE_MAX_AGE=24h
RCLONE_VFS_POLL_INTERVAL=1h
RCLONE_DIR_CACHE_TIME=12h
RCLONE_POLL_INTERVAL=6h
-vv flag
2022/01/20 23:45:31 DEBUG : rclone: Version "v1.57.0" starting with parameters ["/usr/bin/rclone" "mount" "remote:bucket" "/remote" "--umask=0002" "--allow-other" "--default-permissions" "-vv"]
2022/01/20 23:45:31 DEBUG : Creating backend with remote "remote:bucket"
2022/01/20 23:45:31 DEBUG : Using config file from "/etc/rclone.conf"
2022/01/20 23:45:31 INFO : GCS bucket bucket: poll-interval is not supported by this remote
2022/01/20 23:45:31 DEBUG : vfs cache: root is "/var/cache/remote/rclone"
2022/01/20 23:45:31 DEBUG : vfs cache: data root is "/var/cache/remote/rclone/vfs/remote/bucket"
2022/01/20 23:45:31 DEBUG : vfs cache: metadata root is "/var/cache/remote/rclone/vfsMeta/remote/bucket"
2022/01/20 23:45:31 DEBUG : Creating backend with remote "/var/cache/remote/rclone/vfs/remote/bucket"
2022/01/20 23:45:31 DEBUG : Creating backend with remote "/var/cache/remote/rclone/vfsMeta/remote/bucket"
If you want to see what memory is doing in rclone, you need to grab this:
Remote Control / API (rclone.org)
As you mount/explanation seems odd/strange/not right, but who knows, maybe you got a bug.
cberni
January 31, 2022, 8:38pm
3
go tool pprof -text http://localhost:5572/debug/pprof/heap
Fetching profile over HTTP from http://localhost:5572/debug/pprof/heap
Saved profile in /root/pprof/pprof.rclone.alloc_objects.alloc_space.inuse_objects.inuse_space.006.pb.gz
File: rclone
Type: inuse_space
Time: Jan 31, 2022 at 5:35pm (-03)
Showing nodes accounting for 18478.75MB, 98.39% of 18781.27MB total
Dropped 82 nodes (cum <= 93.91MB)
flat flat% sum% cum cum%
6976.95MB 37.15% 37.15% 6976.95MB 37.15% encoding/json.(*decodeState).literalStore
4828.88MB 25.71% 62.86% 4828.88MB 25.71% github.com/rclone/rclone/vfs.newFile
2959.32MB 15.76% 78.62% 3796.34MB 20.21% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).newObjectWithInfo
1948.18MB 10.37% 88.99% 6869.08MB 36.57% github.com/rclone/rclone/vfs.(*Dir)._readDirFromEntries
837.03MB 4.46% 93.45% 837.03MB 4.46% encoding/hex.EncodeToString (inline)
524.85MB 2.79% 96.24% 524.85MB 2.79% bazil.org/fuse/fs.(*Server).saveNode
402.51MB 2.14% 98.38% 412.07MB 2.19% github.com/rclone/rclone/cmd/mount.(*Dir).Lookup
0.54MB 0.0029% 98.39% 18699.96MB 99.57% bazil.org/fuse/fs.(*Server).serve
0.50MB 0.0027% 98.39% 18699.42MB 99.56% bazil.org/fuse/fs.(*Server).handleRequest
0 0% 98.39% 18699.96MB 99.57% bazil.org/fuse/fs.(*Server).Serve.func1
0 0% 98.39% 524.85MB 2.79% bazil.org/fuse/fs.(*Server).saveLookup
0 0% 98.39% 6978.71MB 37.16% encoding/json.(*Decoder).Decode
0 0% 98.39% 6976.95MB 37.15% encoding/json.(*decodeState).array
0 0% 98.39% 6976.95MB 37.15% encoding/json.(*decodeState).object
0 0% 98.39% 6976.95MB 37.15% encoding/json.(*decodeState).unmarshal
0 0% 98.39% 6976.95MB 37.15% encoding/json.(*decodeState).value
0 0% 98.39% 10813.58MB 57.58% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).List
0 0% 98.39% 3833.34MB 20.41% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).itemToDirEntry
0 0% 98.39% 10813.58MB 57.58% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).list
0 0% 98.39% 6980.23MB 37.17% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).list.func1
0 0% 98.39% 10813.58MB 57.58% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).listDir
0 0% 98.39% 3833.34MB 20.41% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).listDir.func1
0 0% 98.39% 837.03MB 4.46% github.com/rclone/rclone/backend/googlecloudstorage.(*Object).setMetaData
0 0% 98.39% 17673.09MB 94.10% github.com/rclone/rclone/cmd/mount.(*Dir).ReadDirAll
0 0% 98.39% 6980.23MB 37.17% github.com/rclone/rclone/fs.pacerInvoker
0 0% 98.39% 10813.58MB 57.58% github.com/rclone/rclone/fs/list.DirSorted
0 0% 98.39% 6980.23MB 37.17% github.com/rclone/rclone/lib/pacer.(*Pacer).Call
0 0% 98.39% 6980.23MB 37.17% github.com/rclone/rclone/lib/pacer.(*Pacer).call
0 0% 98.39% 17673.09MB 94.10% github.com/rclone/rclone/vfs.(*Dir).ReadDirAll
0 0% 98.39% 17682.65MB 94.15% github.com/rclone/rclone/vfs.(*Dir)._readDir
0 0% 98.39% 6978.71MB 37.16% google.golang.org/api/internal/gensupport.DecodeResponse (inline)
0 0% 98.39% 6980.23MB 37.17% google.golang.org/api/storage/v1.(*ObjectsListCall).Do
Neat.
What's the rough number of objects and breakdown of directories/files?
I think it's doing something a very large file, does that make sense?
We'd probably need @ncw as that does seem like quite the memory use!
cberni
February 1, 2022, 12:35am
5
it is a lot of small files.
ex: one month production (December)
the remote backend has more than one year of files.
Right now (at night) there is not much people working and the memory use does not decrease. It works lot of days using just 1% of total memory only. And today I got this 26% of total memory.
Seems to have some trigger/problem that when actives the memory is not released anymore.
If restart the mount the memory get back to 1% in few seconds. (systemctl restart rclone.service)
cberni
February 16, 2022, 2:59pm
6
some problema in another machine and region. This one is all the same configs and versions and the only differ is a union mount (local and remote instead only remote).
Fetching profile over HTTP from http://localhost:5572/debug/pprof/heap
Saved profile in /root/pprof/pprof.rclone.alloc_objects.alloc_space.inuse_objects.inuse_space.001.pb.gz
File: rclone
Type: inuse_space
Time: Feb 15, 2022 at 3:37pm (-03)
Showing nodes accounting for 1106.77MB, 97.86% of 1130.95MB total
Dropped 169 nodes (cum <= 5.65MB)
flat flat% sum% cum cum%
187.53MB 16.58% 16.58% 187.53MB 16.58% encoding/json.(*decodeState).literalStore
117.52MB 10.39% 26.97% 117.52MB 10.39% github.com/rclone/rclone/vfs.newFile
96MB 8.49% 35.46% 96MB 8.49% google.golang.org/api/internal/gensupport.NewMediaBuffer (inline)
65.01MB 5.75% 41.21% 73.01MB 6.46% github.com/rclone/rclone/fs/hash.(*MultiHasher).Sums
64.51MB 5.70% 46.91% 84.01MB 7.43% github.com/rclone/rclone/vfs/vfscache.newItem
58.51MB 5.17% 52.09% 98.51MB 8.71% github.com/rclone/rclone/backend/local.(*Fs).newObject
51.50MB 4.55% 56.64% 51.50MB 4.55% path.Join
46.50MB 4.11% 60.75% 61.04MB 5.40% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).newObjectWithInfo
40MB 3.54% 64.29% 40MB 3.54% strings.(*Builder).grow (inline)
39.01MB 3.45% 67.74% 42.51MB 3.76% github.com/rclone/rclone/fs/hash.StreamTypes
36MB 3.18% 70.92% 36MB 3.18% github.com/rclone/rclone/backend/union.(*Fs).wrapEntries
35.13MB 3.11% 74.03% 35.13MB 3.11% github.com/rclone/rclone/vfs.(*Dir).addObject
33.50MB 2.96% 76.99% 33.50MB 2.96% encoding/hex.EncodeToString (inline)
33.50MB 2.96% 79.95% 33.50MB 2.96% github.com/rclone/rclone/backend/union/upstream.(*Fs).WrapObject (inline)
31.50MB 2.79% 82.74% 31.50MB 2.79% github.com/rclone/rclone/fs.NewDir (inline)
27.50MB 2.43% 85.17% 187.53MB 16.58% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).Put
24.62MB 2.18% 87.35% 116.63MB 10.31% github.com/rclone/rclone/vfs.(*Dir)._readDirFromEntries
22.50MB 1.99% 89.34% 22.50MB 1.99% github.com/rclone/rclone/vfs.newDir
19.50MB 1.72% 91.06% 19.50MB 1.72% sync.NewCond (inline)
17.50MB 1.55% 92.61% 50.04MB 4.42% github.com/rclone/rclone/vfs.newRWFileHandle
13.50MB 1.19% 93.80% 34.50MB 3.05% github.com/rclone/rclone/backend/union.(*Fs).mergeDirEntries
12MB 1.06% 94.86% 12MB 1.06% strings.(*Builder).Write
8.91MB 0.79% 95.65% 92.92MB 8.22% github.com/rclone/rclone/vfs/vfscache.(*Cache)._get
8MB 0.71% 96.36% 23MB 2.03% github.com/rclone/rclone/backend/union.(*Fs).put
6.50MB 0.57% 96.93% 6.50MB 0.57% github.com/rclone/rclone/backend/union/upstream.(*Fs).WrapDirectory (inline)
6MB 0.53% 97.46% 6MB 0.53% github.com/rclone/rclone/vfs.(*File).addWriter
3MB 0.27% 97.73% 166.56MB 14.73% github.com/rclone/rclone/cmd/mount.(*Dir).Create
1.50MB 0.13% 97.86% 151.47MB 13.39% github.com/rclone/rclone/cmd/mount.(*Dir).Lookup
0 0% 97.86% 417.95MB 36.96% bazil.org/fuse/fs.(*Server).Serve.func1
0 0% 97.86% 417.95MB 36.96% bazil.org/fuse/fs.(*Server).handleRequest
0 0% 97.86% 417.95MB 36.96% bazil.org/fuse/fs.(*Server).serve
0 0% 97.86% 187.53MB 16.58% encoding/json.(*Decoder).Decode
0 0% 97.86% 131.02MB 11.58% encoding/json.(*decodeState).array
0 0% 97.86% 188.03MB 16.63% encoding/json.(*decodeState).object
0 0% 97.86% 188.03MB 16.63% encoding/json.(*decodeState).unmarshal
0 0% 97.86% 188.03MB 16.63% encoding/json.(*decodeState).value
0 0% 97.86% 12MB 1.06% fmt.Fprintf
0 0% 97.86% 206.53MB 18.26% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).List
0 0% 97.86% 75.51MB 6.68% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).itemToDirEntry
0 0% 97.86% 206.53MB 18.26% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).list
0 0% 97.86% 131.02MB 11.58% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).list.func1
0 0% 97.86% 206.53MB 18.26% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).listDir
0 0% 97.86% 75.51MB 6.68% github.com/rclone/rclone/backend/googlecloudstorage.(*Fs).listDir.func1
0 0% 97.86% 162.53MB 14.37% github.com/rclone/rclone/backend/googlecloudstorage.(*Object).Update
0 0% 97.86% 153.52MB 13.57% github.com/rclone/rclone/backend/googlecloudstorage.(*Object).Update.func1
0 0% 97.86% 22MB 1.95% github.com/rclone/rclone/backend/googlecloudstorage.(*Object).setMetaData
0 0% 97.86% 108.51MB 9.59% github.com/rclone/rclone/backend/local.(*Fs).List
0 0% 97.86% 109.01MB 9.64% github.com/rclone/rclone/backend/local.(*Fs).Put
0 0% 97.86% 34.50MB 3.05% github.com/rclone/rclone/backend/local.(*Fs).cleanRemote
0 0% 97.86% 40MB 3.54% github.com/rclone/rclone/backend/local.(*Fs).localPath
0 0% 97.86% 60.50MB 5.35% github.com/rclone/rclone/backend/local.(*Fs).newObjectWithInfo
0 0% 97.86% 42.51MB 3.76% github.com/rclone/rclone/backend/local.(*Object).Hash
0 0% 97.86% 73.01MB 6.46% github.com/rclone/rclone/backend/local.(*Object).Update
0 0% 97.86% 34.50MB 3.05% github.com/rclone/rclone/backend/union.(*Fs).List
0 0% 97.86% 343.04MB 30.33% github.com/rclone/rclone/backend/union.(*Fs).List.func1
0 0% 97.86% 23MB 2.03% github.com/rclone/rclone/backend/union.(*Fs).Put
0 0% 97.86% 308.54MB 27.28% github.com/rclone/rclone/backend/union.(*Fs).put.func1
0 0% 97.86% 657.61MB 58.15% github.com/rclone/rclone/backend/union.multithread.func1
0 0% 97.86% 296.54MB 26.22% github.com/rclone/rclone/backend/union/upstream.(*Fs).Put
0 0% 97.86% 28MB 2.48% github.com/rclone/rclone/backend/union/upstream.(*Fs).WrapEntry
0 0% 97.86% 8.56MB 0.76% github.com/rclone/rclone/cmd.Main
0 0% 97.86% 35.50MB 3.14% github.com/rclone/rclone/cmd/mount.(*File).Open
0 0% 97.86% 55.10MB 4.87% github.com/rclone/rclone/cmd/mount.(*FileHandle).Read
0 0% 97.86% 8.06MB 0.71% github.com/rclone/rclone/cmd/mountlib.(*MountPoint).Mount
0 0% 97.86% 8.06MB 0.71% github.com/rclone/rclone/cmd/mountlib.NewMountCommand.func1
0 0% 97.86% 54.51MB 4.82% github.com/rclone/rclone/fs.Fingerprint
0 0% 97.86% 286.08MB 25.30% github.com/rclone/rclone/fs.pacerInvoker
0 0% 97.86% 34.50MB 3.05% github.com/rclone/rclone/fs/list.DirSorted
0 0% 97.86% 23MB 2.03% github.com/rclone/rclone/fs/operations.Copy
0 0% 97.86% 132.55MB 11.72% github.com/rclone/rclone/lib/pacer.(*Pacer).Call
0 0% 97.86% 153.52MB 13.57% github.com/rclone/rclone/lib/pacer.(*Pacer).CallNoRetry (inline)
0 0% 97.86% 286.08MB 25.30% github.com/rclone/rclone/lib/pacer.(*Pacer).call
0 0% 97.86% 47.51MB 4.20% github.com/rclone/rclone/vfs.(*Dir).Create
0 0% 97.86% 151.13MB 13.36% github.com/rclone/rclone/vfs.(*Dir).Stat
0 0% 97.86% 151.13MB 13.36% github.com/rclone/rclone/vfs.(*Dir)._readDir
0 0% 97.86% 151.13MB 13.36% github.com/rclone/rclone/vfs.(*Dir).stat
0 0% 97.86% 151.56MB 13.40% github.com/rclone/rclone/vfs.(*File).Open
0 0% 97.86% 15.50MB 1.37% github.com/rclone/rclone/vfs.(*File).Path
0 0% 97.86% 50.04MB 4.42% github.com/rclone/rclone/vfs.(*File).openRW
0 0% 97.86% 55.10MB 4.87% github.com/rclone/rclone/vfs.(*RWFileHandle).ReadAt
0 0% 97.86% 26.54MB 2.35% github.com/rclone/rclone/vfs.(*RWFileHandle).Truncate
0 0% 97.86% 55.10MB 4.87% github.com/rclone/rclone/vfs.(*RWFileHandle)._readAt
0 0% 97.86% 81.64MB 7.22% github.com/rclone/rclone/vfs.(*RWFileHandle).openPending
0 0% 97.86% 8.06MB 0.71% github.com/rclone/rclone/vfs.(*VFS).SetCacheMode
0 0% 97.86% 8.06MB 0.71% github.com/rclone/rclone/vfs.New
0 0% 97.86% 86.02MB 7.61% github.com/rclone/rclone/vfs/vfscache.(*Cache).Exists
0 0% 97.86% 92.92MB 8.22% github.com/rclone/rclone/vfs/vfscache.(*Cache).get
0 0% 97.86% 8.06MB 0.71% github.com/rclone/rclone/vfs/vfscache.(*Cache).reload
0 0% 97.86% 8.06MB 0.71% github.com/rclone/rclone/vfs/vfscache.(*Cache).reload.func1
0 0% 97.86% 8.06MB 0.71% github.com/rclone/rclone/vfs/vfscache.(*Cache).walk
0 0% 97.86% 8.06MB 0.71% github.com/rclone/rclone/vfs/vfscache.(*Cache).walk.func1
0 0% 97.86% 31MB 2.74% github.com/rclone/rclone/vfs/vfscache.(*Item).Close.func2
0 0% 97.86% 47.01MB 4.16% github.com/rclone/rclone/vfs/vfscache.(*Item).Open
0 0% 97.86% 47.01MB 4.16% github.com/rclone/rclone/vfs/vfscache.(*Item)._checkObject
0 0% 97.86% 31MB 2.74% github.com/rclone/rclone/vfs/vfscache.(*Item)._store
0 0% 97.86% 7.50MB 0.66% github.com/rclone/rclone/vfs/vfscache.(*Item)._updateFingerprint
0 0% 97.86% 47.01MB 4.16% github.com/rclone/rclone/vfs/vfscache.(*Item).open
0 0% 97.86% 31MB 2.74% github.com/rclone/rclone/vfs/vfscache.(*Item).store
0 0% 97.86% 8.06MB 0.71% github.com/rclone/rclone/vfs/vfscache.New
0 0% 97.86% 31MB 2.74% github.com/rclone/rclone/vfs/vfscache/writeback.(*WriteBack).upload
0 0% 97.86% 8.56MB 0.76% github.com/spf13/cobra.(*Command).Execute (inline)
0 0% 97.86% 8.56MB 0.76% github.com/spf13/cobra.(*Command).ExecuteC
0 0% 97.86% 8.56MB 0.76% github.com/spf13/cobra.(*Command).execute
0 0% 97.86% 187.53MB 16.58% google.golang.org/api/internal/gensupport.DecodeResponse (inline)
0 0% 97.86% 96MB 8.49% google.golang.org/api/internal/gensupport.NewInfoFromMedia
0 0% 97.86% 96MB 8.49% google.golang.org/api/internal/gensupport.PrepareUpload
0 0% 97.86% 57.52MB 5.09% google.golang.org/api/storage/v1.(*ObjectsInsertCall).Do
0 0% 97.86% 96MB 8.49% google.golang.org/api/storage/v1.(*ObjectsInsertCall).Media
0 0% 97.86% 131.02MB 11.58% google.golang.org/api/storage/v1.(*ObjectsListCall).Do
0 0% 97.86% 7.91MB 0.7% main.main
0 0% 97.86% 40MB 3.54% path/filepath.Join (inline)
0 0% 97.86% 8.06MB 0.71% path/filepath.Walk
0 0% 97.86% 40MB 3.54% path/filepath.join
0 0% 97.86% 8.06MB 0.71% path/filepath.walk
0 0% 97.86% 5.68MB 0.5% runtime.doInit
0 0% 97.86% 13.58MB 1.20% runtime.main
0 0% 97.86% 40MB 3.54% strings.(*Builder).Grow (inline)
0 0% 97.86% 40MB 3.54% strings.Join
goroutine profile: total 10
1 @ 0x40b8f4 0x4657f8 0x75ecf9 0x469181
# 0x4657f7 os/signal.signal_recv+0x97 runtime/sigqueue.go:169
# 0x75ecf8 os/signal.loop+0x18 os/signal/signal_unix.go:24
1 @ 0x4382f6 0x40640c 0x405e38 0x75f38f 0x469181
# 0x75f38e github.com/rclone/rclone/lib/atexit.Register.func1.1+0x2e github.com/rclone/rclone/lib/atexit/atexit.go:45
1 @ 0x4382f6 0x4309fe 0x463589 0x4db212 0x4dc57a 0x4dc568 0x539769 0x54ca25 0x6f387f 0x469181
# 0x463588 internal/poll.runtime_pollWait+0x88 runtime/netpoll.go:229
# 0x4db211 internal/poll.(*pollDesc).wait+0x31 internal/poll/fd_poll_runtime.go:84
# 0x4dc579 internal/poll.(*pollDesc).waitRead+0x259 internal/poll/fd_poll_runtime.go:89
# 0x4dc567 internal/poll.(*FD).Read+0x247 internal/poll/fd_unix.go:167
# 0x539768 net.(*netFD).Read+0x28 net/fd_posix.go:56
# 0x54ca24 net.(*conn).Read+0x44 net/net.go:183
# 0x6f387e net/http.(*connReader).backgroundRead+0x3e net/http/server.go:672
1 @ 0x4382f6 0x4309fe 0x463589 0x4db212 0x4dc57a 0x4dc568 0x539769 0x54ca25 0x909bf2 0x651a3d 0x507c98 0x651c25 0x64f292 0x654b2f 0x654b30 0x561d94 0x4993da 0x6c0f0e 0x6c0ec8 0x6c1655 0x6e12e5 0x6e08d9 0x469181
# 0x463588 internal/poll.runtime_pollWait+0x88 runtime/netpoll.go:229
# 0x4db211 internal/poll.(*pollDesc).wait+0x31 internal/poll/fd_poll_runtime.go:84
# 0x4dc579 internal/poll.(*pollDesc).waitRead+0x259 internal/poll/fd_poll_runtime.go:89
# 0x4dc567 internal/poll.(*FD).Read+0x247 internal/poll/fd_unix.go:167
# 0x539768 net.(*netFD).Read+0x28 net/fd_posix.go:56
# 0x54ca24 net.(*conn).Read+0x44 net/net.go:183
# 0x909bf1 github.com/rclone/rclone/fs/fshttp.(*timeoutConn).Read+0x31 github.com/rclone/rclone/fs/fshttp/dialer.go:104
# 0x651a3c crypto/tls.(*atLeastReader).Read+0x3c crypto/tls/conn.go:777
# 0x507c97 bytes.(*Buffer).ReadFrom+0x97 bytes/buffer.go:204
# 0x651c24 crypto/tls.(*Conn).readFromUntil+0xe4 crypto/tls/conn.go:799
# 0x64f291 crypto/tls.(*Conn).readRecordOrCCS+0x111 crypto/tls/conn.go:606
# 0x654b2e crypto/tls.(*Conn).readRecord+0x16e crypto/tls/conn.go:574
# 0x654b2f crypto/tls.(*Conn).Read+0x16f crypto/tls/conn.go:1277
# 0x561d93 bufio.(*Reader).Read+0x1b3 bufio/bufio.go:227
# 0x4993d9 io.ReadAtLeast+0x99 io/io.go:328
# 0x6c0f0d io.ReadFull+0x6d io/io.go:347
# 0x6c0ec7 net/http.http2readFrameHeader+0x27 net/http/h2_bundle.go:1553
# 0x6c1654 net/http.(*http2Framer).ReadFrame+0x94 net/http/h2_bundle.go:1811
# 0x6e12e4 net/http.(*http2clientConnReadLoop).run+0x164 net/http/h2_bundle.go:8437
# 0x6e08d8 net/http.(*http2ClientConn).readLoop+0x78 net/http/h2_bundle.go:8359
1 @ 0x4382f6 0x4309fe 0x463589 0x4db212 0x4de56c 0x4de559 0x53af15 0x556e48 0x555cdd 0x6fe594 0x7f7718 0x469181
# 0x463588 internal/poll.runtime_pollWait+0x88 runtime/netpoll.go:229
# 0x4db211 internal/poll.(*pollDesc).wait+0x31 internal/poll/fd_poll_runtime.go:84
# 0x4de56b internal/poll.(*pollDesc).waitRead+0x22b internal/poll/fd_poll_runtime.go:89
# 0x4de558 internal/poll.(*FD).Accept+0x218 internal/poll/fd_unix.go:402
# 0x53af14 net.(*netFD).accept+0x34 net/fd_unix.go:173
# 0x556e47 net.(*TCPListener).accept+0x27 net/tcpsock_posix.go:140
# 0x555cdc net.(*TCPListener).Accept+0x3c net/tcpsock.go:262
# 0x6fe593 net/http.(*Server).Serve+0x393 net/http/server.go:3001
# 0x7f7717 github.com/rclone/rclone/cmd/serve/httplib.(*Server).Serve.func1+0x317 github.com/rclone/rclone/cmd/serve/httplib/httplib.go:353
1 @ 0x4382f6 0x4480d2 0x14c2afb 0x469181
# 0x14c2afa github.com/rclone/rclone/vfs/vfscache.(*Cache).cleaner+0x17a github.com/rclone/rclone/vfs/vfscache/cache.go:784
1 @ 0x4382f6 0x4480d2 0x14ec28a 0x14eb8f8 0x146c118 0x146c81c 0x147a076 0x147a06b 0x15e9957 0x437f27 0x469181
# 0x14ec289 github.com/rclone/rclone/cmd/mountlib.(*MountPoint).Wait+0x249 github.com/rclone/rclone/cmd/mountlib/mount.go:290
# 0x14eb8f7 github.com/rclone/rclone/cmd/mountlib.NewMountCommand.func1+0x297 github.com/rclone/rclone/cmd/mountlib/mount.go:181
# 0x146c117 github.com/spf13/cobra.(*Command).execute+0x5f7 github.com/spf13/cobra@v1.2.1/command.go:860
# 0x146c81b github.com/spf13/cobra.(*Command).ExecuteC+0x3bb github.com/spf13/cobra@v1.2.1/command.go:974
# 0x147a075 github.com/spf13/cobra.(*Command).Execute+0x75 github.com/spf13/cobra@v1.2.1/command.go:902
# 0x147a06a github.com/rclone/rclone/cmd.Main+0x6a github.com/rclone/rclone/cmd/cmd.go:555
# 0x15e9956 main.main+0x16 github.com/rclone/rclone/rclone.go:14
# 0x437f26 runtime.main+0x226 runtime/proc.go:255
1 @ 0x4382f6 0x4480d2 0xc1c099 0x469181
# 0xc1c098 go.opencensus.io/stats/view.(*worker).start+0xb8 go.opencensus.io@v0.23.0/stats/view/worker.go:276
1 @ 0x4630a5 0x783a35 0x78384d 0x7809cb 0x78f5da 0x79018e 0x6fa5ef 0x6fc3a9 0x6fe01b 0x6f96c8 0x469181
# 0x4630a4 runtime/pprof.runtime_goroutineProfileWithLabels+0x24 runtime/mprof.go:746
# 0x783a34 runtime/pprof.writeRuntimeProfile+0xb4 runtime/pprof/pprof.go:724
# 0x78384c runtime/pprof.writeGoroutine+0x4c runtime/pprof/pprof.go:684
# 0x7809ca runtime/pprof.(*Profile).WriteTo+0x14a runtime/pprof/pprof.go:331
# 0x78f5d9 net/http/pprof.handler.ServeHTTP+0x499 net/http/pprof/pprof.go:253
# 0x79018d net/http/pprof.Index+0x12d net/http/pprof/pprof.go:371
# 0x6fa5ee net/http.HandlerFunc.ServeHTTP+0x2e net/http/server.go:2046
# 0x6fc3a8 net/http.(*ServeMux).ServeHTTP+0x148 net/http/server.go:2424
# 0x6fe01a net/http.serverHandler.ServeHTTP+0x43a net/http/server.go:2878
# 0x6f96c7 net/http.(*conn).serve+0xb07 net/http/server.go:1929
1 @ 0x481225 0x47e4ed 0x1498546 0x14984aa 0x14ac155 0x14f63be 0x469181
# 0x481224 syscall.Syscall+0x4 syscall/asm_linux_amd64.s:20
# 0x47e4ec syscall.read+0x4c syscall/zsyscall_linux_amd64.go:687
# 0x1498545 syscall.Read+0xc5 syscall/syscall_unix.go:189
# 0x14984a9 bazil.org/fuse.(*Conn).ReadRequest+0x29 bazil.org/fuse@v0.0.0-20200524192727-fb710f7dfd05/fuse.go:580
# 0x14ac154 bazil.org/fuse/fs.(*Server).Serve+0x3d4 bazil.org/fuse@v0.0.0-20200524192727-fb710f7dfd05/fs/serve.go:501
# 0x14f63bd github.com/rclone/rclone/cmd/mount.mount.func2+0x3d github.com/rclone/rclone/cmd/mount/mount.go:100
cberni
February 16, 2022, 3:00pm
7
I found the docs about --use-mmap.
I change to use it right now... hope can help.
cberni
February 17, 2022, 5:12pm
8
still using lot of memory.
any other ideia?
system
March 19, 2022, 5:12pm
9
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.