Cache problem after powerloss

Help and Support
1

My RPI4 lost power in a storm, after restarting, I'm getting errors about not being able to find/access a file in the cache, even though it looks like the file was there right after the reboot (there being in the cache/vfs folder). But now it's gone.

My rpi4 (Fedora33-5.12.14-200.fc33.aarch64) running rclone:
rclone v1.55.1

  • os/type: linux
  • os/arch: arm64
  • go/version: go1.16.3
  • go/linking: static

Connecting to Googledrive

Errors:>

Jul 08 11:00:55 localhost rclone[1442]: ERROR : VirtualBox VMs/XpPlay/XpPlay.vdi: Failed to copy: Post "https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks&supportsAllDrives=true&uploadType=resumable&upload_id=ADPycdtZnuHOc9CJ1S8rH8FuNGgUOmNO9i1QKGaWGgrUzqqCM52w9Da6fer7DY4ox3kvEcINTaWFYgPfrOxXu4raJQ": context canceled
Jul 08 11:00:55 localhost rclone[1442]: INFO : VirtualBox VMs/XpPlay/XpPlay.vdi: vfs cache: upload canceled
Jul 08 11:05:07 localhost rclone[2313]: ERROR : GenJ/LaptopElex/1480047384923.jpg: vfs cache: failed to reload item: reload: failed to add virtual dir entry: file does not exist
Jul 08 11:05:07 localhost rclone[2313]: INFO : GenJ/LaptopElex/1481033725014.jpg: vfs cache: queuing for upload in 5s
Jul 08 14:28:23 localhost rclone[5042]: ERROR : GenJ/RewardingFrequentBunting-mobile.mp4: WriteFileHandle: Can't open for write without O_TRUNC on existing file without --vfs-cache-mode >= writes
Jul 08 14:28:50 localhost rclone[5042]: ERROR : GenJ/JointDearestMyotis-mobile.mp4: WriteFileHandle: ReadAt: Can't read and write to file without --vfs-cache-mode >= minimal

Rclone commad:

/usr/bin/rclone mount privdrive:/ /sharedata/privdrive
--allow-other
--user-agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.92 Safari/537.36'
--config /tools/rclone/rclone.conf
--use-mmap
--cache-dir /tools/rclone/pcache
--cache-db-path /tools/rclone/pcachedb
--uid=1001
--gid=1001
--dir-cache-time 168h
--poll-interval=1m
--timeout 1h
--umask 000
--file-perms 0777
--vfs-cache-mode writes
--drive-chunk-size=64M
--vfs-cache-max-size=10G
--tpslimit 10
--tpslimit-burst 10
--log-level INFO

Restarting a couple times seems to have cleared them up, but now I'm worried about corruption or how to best handle it in the future.

2

hello,
which cache are you writing about, as there seem to be two different caches?

  • the cache remote
  • the vfs cache

cannot know for sure, as there is no config file.....

3

I was speaking of the cache-dir directory. The file was located after reboot at:
/tools/rclone/pcache/vfs/privdrive/GenJ/LaptopElex/1480047384923.jpg

rclone.conf:

[rct]
type = drive
client_id = XXXX
client_secret = XXXX
scope = drive
acknowledge_abuse = true
token = {"access_token":"XXXX"}
root_folder_id = XXXX

[privdriveraw]
type = crypt
remote = rct:/privdrive
filename_encryption = standard
directory_name_encryption = true
password = XXXX
password2 = XXXX

[privdrive]
type = chunker
remote = privdriveraw:
hash_type = sha1

4

please re-edit that post and redact
client_id
client_secret
root_folder_id

5

I replaced those with garbage strings for the values, but i'm happy to redact.

6

i do not know what is going on when using the chunker remote which has beta status.
chunker -> crypt -> gdrive.

--cache-db-path can be removed.

7

hi,
i am curious, as for the need for chunker, when the largest file size permitted in gdrive is 5TB?

8

In my case, my upload speed is slow enough that if I don't use chunker, uploads exceed their timelimit on the google side for larger files (>10gb).

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.