Cache Remote Keeps Breaking

Help and Support
1

What is the problem you are having with rclone?

cache remote usually works fine for a few months, and then something pops up that makes it break and requires rebuilding the database. i've inlucded the end of the log below, which is where the issue is. for some reason rclone can't delete the files in three directories so the directories can be deleted.

Run the command 'rclone version' and share the full output of the command.

rclone v1.62.2

  • os/version: Microsoft Windows Server 2019 Datacenter 1809 (64 bit)
  • os/kernel: 10.0.17763.3887 Build 17763.3887.3887 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.20.2
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Microsoft SharePoint

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync --verbose --progress -vv "E:\Aleah" "ServerSharePointBackupnewcache:Aleah" --log-file=C:\Users\Administrator\Desktop\Aleah.txt

The rclone config contents with secrets removed.

[ServerSharePoint]
type = onedrive
token = {"access_token":"
drive_id =
drive_type = documentLibrary

[ServerSharePointBackupnew]
type = crypt
remote = ServerSharePoint:SL Server
directory_name_encryption = false
password = lol

[ServerSharePointBackupnewcache]
type = cache
remote = ServerSharePointBackupnew:
chunk_size = 1Gi
info_age = 1y
db_wait_time = 0s

A log from the command with the -vv flag

2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/tl06gdkj1kuccqkgr9d19r891s: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/tls8fenu3tlqn73g1l0sest7fk: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/toes8lp5cfsnrfjqae4hnt518s: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/tshfeljubaav5bo9i6bias0vk4v1pm3654t86tb8rakihi9pog873pp60nmhg0h5uant25fl9823a: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/tur3494575ofeujaifm1j7lils: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/u8gc4refe3bjj8bgnu53kiuoc6ev6uh6ver7aki80g30o1lk77dm942kngcld6ksrrcab5t9gp3ahqb55tl384hsk9irpu0afsqgjno: Skipping undecryptable file name: bad PKCS#7 padding - too short
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/uee24638smjqq4i2l6hhm6qj1c: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/uq9bd8bqj1drtjk0sdh41n7uhovk7l575bchf38aql011404bugsfgbqle7gd53c5omh0pdn9lgf6: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/v4cffn4fbnl27velmb3imac0gg: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/vcnhpjalhjj32l6l0ujao6f535grts29qt33iap58ui920efvufr786p96rp4ukhpdnqtl4t3h1g6: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/vh3sdil4rbc76859cvd8cn0n3s: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/vha4iafd36ot4626jfdjuc0260o1m9msm91iutpjp921v9p36uc30rlv7f8h2lq9grpcaf9oe22r4: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/vhv1ec667ij2qa0s3ssvdsocbcpk599fjto1mgeo3p4djajot4ks2e8r7jdhlnlain3m63hsgo2nb8qug6joj1fi9f3jli94n6sr6kg: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/vipskrvnsee83b3laau8j207ts: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3/vv64ccbm5146rtrhmb1u4h371o1u3bpln9qv70f0tscticmt297mvnr8qoodg8fuie6jt77033i40: Skipping undecryptable file name: bad PKCS#7 padding - too long
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3: list: read 0 from source
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3: list: source entries: []
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3: list: cached directories: 0
2023/03/27 13:18:39 DEBUG : Project/Cookweek #3: list: cached dir: 'Aleah/Project/Cookweek #3', cache ts: 2023-03-27 13:18:39.9143861 -0500 CDT m=+5.727618701
2023/03/27 13:18:39 DEBUG : Cache remote ServerSharePointBackupnewcache:Aleah: Waiting for checks to finish
2023/03/27 13:18:39 DEBUG : Cache remote ServerSharePointBackupnewcache:Aleah: Waiting for transfers to finish
2023/03/27 13:18:39 DEBUG : Waiting for deletions to finish
2023/03/27 13:18:39 INFO  : Project/cookweek #7: Removing directory
2023/03/27 13:18:39 DEBUG : Cache remote ServerSharePointBackupnewcache:Aleah: rmdir 'Project/cookweek #7'
2023/03/27 13:18:40 DEBUG : Project/cookweek #7: Failed to Rmdir: directory not empty
2023/03/27 13:18:40 INFO  : Project/cokweek #6: Removing directory
2023/03/27 13:18:40 DEBUG : Cache remote ServerSharePointBackupnewcache:Aleah: rmdir 'Project/cokweek #6'
2023/03/27 13:18:40 DEBUG : Project/cokweek #6: Failed to Rmdir: directory not empty
2023/03/27 13:18:40 INFO  : Project/Cookweek #3: Removing directory
2023/03/27 13:18:40 DEBUG : Cache remote ServerSharePointBackupnewcache:Aleah: rmdir 'Project/Cookweek #3'
2023/03/27 13:18:42 DEBUG : Project/Cookweek #3: Failed to Rmdir: directory not empty
2023/03/27 13:18:42 DEBUG : Cache remote ServerSharePointBackupnewcache:Aleah: failed to delete 3 directories
2023/03/27 13:18:42 INFO  : There was nothing to transfer
2023/03/27 13:18:42 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:             17038 / 17038, 100%
Deleted:                0 (files), 3 (dirs)
Elapsed time:         8.0s

2023/03/27 13:18:42 DEBUG : 9 go routines active
2023/03/27 13:18:42 DEBUG : Cache remote ServerSharePointBackupnewcache:Aleah: Services stopped
2

hi @jared,

the cache remote has been deprecated.
there are outstanding bugs which aren't getting fixed.

based on your config,
ServerSharePointBackupnew: is equivalent to ServerSharePointBackupnewcache:
maybe try that with --dry-run

3

no, one is a crypt remote, and one is a cache of said crypt remote

i know i'm using an outdated remote, kinda.....but we almost can't use rclone without it due to microsoft's stringent I/O limit on the SharePoint servers. i'm not sure it matters, but there's currently no outstanding bugs on it :slight_smile:

4

There are plenty. They got closed out because it's deprecated. I'd really really avoid using it.

5

at present, is there no other way to do a sync without scanning the entire remote directory?

6

Not that I'm aware of as sync needs to check and ensure things are consistent.

7

might try filters, such as --max-age, which work on the source

8

You can mount your remote with the modern vfs-cache then do a local sync to that. It’s a bit of a workaround but gets you a modern, robust, cache and doesn’t rely on the deprecated one.

You may be able to run against the rc interface too but I am less certain

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.