Is parsing through excluded directories normal behavior?

nle · January 15, 2021, 1:38pm

What is the problem you are having with rclone?

Is it normal behavior for rclone to parse through directories that are excluded.
I have excluded my plex directory from my backup because of the great number of files, but rclone still goes through it.

That seems like a bit of a waste?

Example from my filter_rules file:

- /ssdpool/docker/plex/**
+ /**

What is your rclone version (output from `rclone version`)

rclone v1.53.3

os/arch: linux/amd64
go version: go1.15.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Linux Ubuntu 20.04.1 LTS (Focal Fossa)

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone sync …

2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/cf723960e81a1c730420ab471993cd85df5cb2c0: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/cd23277de9061b065a7cc6300fbb0f6bd0fd390d: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/b479b5ab93d6844037e521491ae142b9708fc0ef: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/9a55c1c7b3c5878fab650cc04f1f52e1a36995e5: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/94cbaba70bc8e8457862a3cf4ee54b5418600175: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/45e5fc82ab178de940cce2bbd4b7809203208db6: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/3bba57874baa9e0e5f530e58fe2689e982f0d82e: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/37ed659586757fc593ce7bebf8cf5bad21102a06: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/295c96a836914d812f06793eb00cce8b0b52dc28: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/1cdf61f6ad1e7d8ae18500a728fd82771ec50149: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/194ab3211af80f0b1389d0aabeab4ed6ef37cb78: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Albums/9/3f6665a2d19b74decc03d2d77e35ffe0fd1070b.bundle/Contents/com.plexapp.agents.lyricfind/tracks/052d288a6d8347cd4aca7de73e666b3d45007ab0: Excluded from sync (and deletion)
2021/01/15 14:30:16 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Metadata/Movies/8/fc596d83dc3191b40b81224ed5c30960100b7aa.bundle/Contents/_stored/art/com.plexapp.agents.imdb_cd6fcf7aa645dd8ae9bbf03f6bb69de835c59f7a: Excluded from sync (and deletion)

Animosity022 · January 15, 2021, 2:37pm

Yes, it's normal as it has to get a directory list to know that it will be excluded so you are seeing the output of that.

nle · January 15, 2021, 3:50pm

ok, thanks, but isn't that a bit ineffective and unessesary to dig further down in the directory structure since I've explicitly said it's excluded?

Animosity022 · January 15, 2021, 3:53pm

It's a chicken and egg thing.

It has to get a directory listing to know the directory is there and is excluded. Since it pulls the directory listing, you are seeing the extra debug output that it's being excluded.

You can't exclude it if you don't know it's there.

calisro · January 15, 2021, 4:09pm

I think the important bit here is that rclone matches on files and not directories. So your excludes/includes need to look at the files contained to know to exclude them.

nle · January 15, 2021, 4:10pm

I'm not sure if I follow completely.

In this case I'm backing up my docker data, and want to exclude plex because that directory contains 308701 directories, 303508 files.

My thinking is, when rclone is reading - /ssdpool/docker/plex/**, that should be enough for rclone to not list anything further down than /ssdpool/docker/plex.

Why would it, since I've said it shouldn't go there. If I explicitly added something further down in the same path ie. /ssdpool/docker/plex/another/dir/**, that would make sense to index that path.

Maybe the development cost to keep rclone out, isn't worth it. But it takes quite some time to go through, and that is on a SSD zmirror pool. And I'm guessing there is a lot of use cases with a lot more files than this.

Animosity022 · January 15, 2021, 4:13pm

Can you share a full log as I ran a quick test and I get:

felix@gemini:~$ rclone sync /var/lib/plexmediaserver/ /data/test --filter-from filter -vv --dry-run
2021/01/15 11:11:47 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "/var/lib/plexmediaserver/" "/data/test" "--filter-from" "filter" "-vv" "--dry-run"]
2021/01/15 11:11:47 DEBUG : Creating backend with remote "/var/lib/plexmediaserver/"
2021/01/15 11:11:47 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2021/01/15 11:11:47 DEBUG : Creating backend with remote "/data/test"
2021/01/15 11:11:47 DEBUG : Library: Excluded
2021/01/15 11:11:47 DEBUG : Local file system at /data/test: Waiting for checks to finish
2021/01/15 11:11:47 DEBUG : Local file system at /data/test: Waiting for transfers to finish
2021/01/15 11:11:47 NOTICE: blah: Skipped copy as --dry-run is set
2021/01/15 11:11:47 DEBUG : Waiting for deletions to finish
2021/01/15 11:11:47 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.0s

2021/01/15 11:11:47 DEBUG : 3 go routines active
felix@gemini:~$ ls /var/lib/plexmediaserver/
Library  blah
felix@gemini:~$ cat filter
- /Library/**
+ /**

and without dry-run

felix@gemini:~$ rclone sync /var/lib/plexmediaserver/ /data/test --filter-from filter -vv
2021/01/15 11:12:56 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "/var/lib/plexmediaserver/" "/data/test" "--filter-from" "filter" "-vv"]
2021/01/15 11:12:56 DEBUG : Creating backend with remote "/var/lib/plexmediaserver/"
2021/01/15 11:12:56 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2021/01/15 11:12:56 DEBUG : Creating backend with remote "/data/test"
2021/01/15 11:12:56 DEBUG : Library: Excluded
2021/01/15 11:12:56 DEBUG : Local file system at /data/test: Waiting for checks to finish
2021/01/15 11:12:56 DEBUG : Local file system at /data/test: Waiting for transfers to finish
2021/01/15 11:12:56 DEBUG : blah: MD5 = d41d8cd98f00b204e9800998ecf8427e OK
2021/01/15 11:12:56 INFO  : blah: Copied (new)
2021/01/15 11:12:56 DEBUG : Waiting for deletions to finish
2021/01/15 11:12:56 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.0s

2021/01/15 11:12:56 DEBUG : 2 go routines active
felix@gemini:~$ ls /data/test
blah

nle · January 15, 2021, 4:45pm

Strange. That is what's logical.

I'm currently running the backup, and the log file is extremely huge atm. But I'll have a closer look and see if I spot anything off.

I did also a test here, and I'm seeing the same result as you when testing. I tested with the same flags and same remote as the one with the issue.

It makes no sense, since you can see in my original log excerpt that rclone recognize that the files is excluded.

Test

Filter rules used in the test (same as the original, except less excluded paths and a bit shorter paths since I'm running it from a different directory)

- .Spotlight-*/**
- .Spotlight-V100/**
- .TemporaryItems/**
- .Trashes/**
- .Trash/**
- .$EXTEND/**
- .AppleDouble/**
- *Cache*/**

- *.idlk
- .DS_Store
- ._.DS_Store
- .metadata
- .localized
- .com.apple.timemachine.supported
- ._*
- *.$EXTEND
- Icon\r

- Library/**
- transcode/**
+ /**

rclone sync . gdrive-crypt:test --filter-from filter --create-empty-src-dirs --transfers 16 --checkers 16 --drive-chunk-size 128M --fast-list --max-transfer 725G --modify-window 1ms -vv --dry-run
2021/01/15 17:34:26 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "." "gdrive-crypt:test" "--filter-from" "filter" "--create-empty-src-dirs" "--transfers" "16" "--checkers" "16" "--drive-chunk-size" "128M" "--fast-list" "--max-transfer" "725G" "--modify-window" "1ms" "- "-vv" "--dry-run"]
2021/01/15 17:34:26 DEBUG : Creating backend with remote "."
2021/01/15 17:34:26 DEBUG : Using config file from "/mnt/<host>/datapool/backup/_rclone/gsuite/rclone.conf"
2021/01/15 17:34:26 DEBUG : fs cache: renaming cache item "." to be canonical "/mnt/frostbite/ssdpool/docker/plex"
2021/01/15 17:34:26 DEBUG : Creating backend with remote "gdrive-crypt:test"
2021/01/15 17:34:26 DEBUG : Creating backend with remote "gdrive:/r05sdmm8khmcldvqapvh5pou2c"
2021/01/15 17:34:27 DEBUG : fs cache: renaming cache item "gdrive:/r05sdmm8khmcldvqapvh5pou2c" to be canonical "gdrive:r05sdmm8khmcldvqapvh5pou2c"
2021/01/15 17:34:27 DEBUG : fs cache: switching user supplied name "gdrive:/r05sdmm8khmcldvqapvh5pou2c" for canonical name "gdrive:r05sdmm8khmcldvqapvh5pou2c"
2021/01/15 17:34:27 DEBUG : Library: Excluded
2021/01/15 17:34:27 DEBUG : transcode: Excluded
2021/01/15 17:34:27 NOTICE: filter: Skipped copy as --dry-run is set
2021/01/15 17:34:27 DEBUG : Encrypted drive 'gdrive-crypt:test': Waiting for checks to finish
2021/01/15 17:34:27 DEBUG : Encrypted drive 'gdrive-crypt:test': Waiting for transfers to finish
2021/01/15 17:34:27 NOTICE: blah: Skipped make directory as --dry-run is set
2021/01/15 17:34:27 DEBUG : Encrypted drive 'gdrive-crypt:test': copied 1 directories
2021/01/15 17:34:27 DEBUG : Waiting for deletions to finish
2021/01/15 17:34:27 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         1.3s

2021/01/15 17:34:27 DEBUG : 4 go routines active

nle · January 15, 2021, 4:58pm

I can't see why this would be any different, but hopefully there is a stupid thing in here somewhere.

I'm running this with --dry-run, but I get the same issue as the orignal post.

Command

rclone sync /mnt/<host> gdrive-crypt:/last_snapshot --backup-dir=gdrive-crypt:/archive/2021/2021-01-15_10:31:45 --config /mnt/<host>/datapool/backup/_rclone/gsuite/rclone.conf --filter-from /mnt/<host>/datapool/backup/_rclone/gsuite/filter_rules --transfers 16 --checkers 16 --drive-chunk-size 128M --create-empty-src-dirs --fast-list --max-transfer 725G --modify-window 1ms --user-agent PqJiMsNL2pkPYkLUR7ttBYUuTnPDz4Pt7gkNACs --log-level DEBUG --bwlimit '08:00,50M 23:50,off' --dry-run

Filter rules

- .Spotlight-*/**
- .Spotlight-V100/**
- .TemporaryItems/**
- .Trashes/**
- .Trash/**
- .$EXTEND/**
- .AppleDouble/**
- *Cache*/**

- *.idlk
- .DS_Store
- ._.DS_Store
- .metadata
- .localized
- .com.apple.timemachine.supported
- ._*
- *.$EXTEND
- Icon\r

- /datapool/backup/ESXi/VM_Backup/**
- /datapool/backup/_rclone/*/log/**

- /datapool/docker2/**
- /datapool/download/**
- /datapool/storage/**
- /datapool/sync/**
- /datapoo/test/**
- /datapool/timemachine/**
- /datapool/vault/_ekskludert/**
- /datapool/vm/**

- /ssdpool/docker/plex/**
- /ssdpool/docker/bookstack/**
- /ssdpool/docker/_cache/**

+ /**

(I also tried without the starting slash on the filters ie. - ssdpool/docker/plex/** – but did not change anything)

Log excerpt

rclone sync /mnt/<host> gdrive-crypt:/last_snapshot --backup-dir=gdrive-crypt:/archive/2021/2021-01-15_10:31:45 --config /mnt/<host>/datapool/backup/_rclone/gsuite/rclo
ne.conf --filter-from /mnt/<host>/datapool/backup/_rclone/gsuite/filter_rules --transfers 16 --checkers 16 --drive-chunk-size 128M --create-empty-src-dirs --fast-list --max-transfer 725G --modify-window 1ms --user-agent PqJi
MsNL2pkPYkLUR7ttBYUuTnPDz4Pt7gkNACs --log-level DEBUG --bwlimit '08:00,50M 23:50,off' --dry-run
2021/01/15 17:49:54 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "/mnt/<host>" "gdrive-crypt:/last_snapshot" "--backup-dir=gdrive-crypt:/archive/2021/2021-01-15_10:31:45" "--config" "/mnt/<host>/datapool/backup/_rclone/gsuite/rclone.conf" "--filter-from" "/mnt/<host>/datapool/backup/_rclone/gsuite/filter_rules" "--transfers" "16" "--checkers" "16" "--drive-chunk-size" "128M" "--create-empty-src-dirs" "--fast-li
st" "--max-transfer" "725G" "--modify-window" "1ms" "--user-agent" "PqJiMsNL2pkPYkLUR7ttBYUuTnPDz4Pt7gkNACs" "--log-level" "DEBUG" "--bwlimit" "08:00,50M 23:50,off" "--dry-run"]
2021/01/15 17:49:54 DEBUG : Creating backend with remote "/mnt/<host>"
2021/01/15 17:49:54 DEBUG : Using config file from "/mnt/<host>/datapool/backup/_rclone/gsuite/rclone.conf"
2021/01/15 17:49:54 INFO  : Starting bandwidth limiter at 50MBytes/s
2021/01/15 17:49:54 DEBUG : Creating backend with remote "gdrive-crypt:/last_snapshot"
2021/01/15 17:49:55 DEBUG : Creating backend with remote "gdrive:/8tjka0h7c3rtgbo899cicheqgg"
2021/01/15 17:49:55 DEBUG : fs cache: renaming cache item "gdrive:/8tjka0h7c3rtgbo899cicheqgg" to be canonical "gdrive:8tjka0h7c3rtgbo899cicheqgg"
2021/01/15 17:49:55 DEBUG : fs cache: switching user supplied name "gdrive:/8tjka0h7c3rtgbo899cicheqgg" for canonical name "gdrive:8tjka0h7c3rtgbo899cicheqgg"
2021/01/15 17:49:55 DEBUG : Creating backend with remote "gdrive-crypt:/archive/2021/2021-01-15_10:31:45"
2021/01/15 17:49:55 DEBUG : Creating backend with remote "gdrive:/acf7h4pgat85n668rtee7l0888/5ca186ppoaroc4jmj6qn12utmo/2558csgag091c4l0k9ke76qfdm3j5rje9lcnccjivgeh0pkhg88g"
2021/01/15 17:49:59 DEBUG : fs cache: renaming cache item "gdrive:/acf7h4pgat85n668rtee7l0888/5ca186ppoaroc4jmj6qn12utmo/2558csgag091c4l0k9ke76qfdm3j5rje9lcnccjivgeh0pkhg88g" to be canonical "gdrive:acf7h4pgat85n668rtee7l0888/5
ca186ppoaroc4jmj6qn12utmo/2558csgag091c4l0k9ke76qfdm3j5rje9lcnccjivgeh0pkhg88g"
2021/01/15 17:49:59 DEBUG : fs cache: switching user supplied name "gdrive:/acf7h4pgat85n668rtee7l0888/5ca186ppoaroc4jmj6qn12utmo/2558csgag091c4l0k9ke76qfdm3j5rje9lcnccjivgeh0pkhg88g" for canonical name "gdrive:acf7h4pgat85n668
rtee7l0888/5ca186ppoaroc4jmj6qn12utmo/2558csgag091c4l0k9ke76qfdm3j5rje9lcnccjivgeh0pkhg88g"
2021/01/15 17:50:00 DEBUG : ssdpool/docker/plex: Excluded from sync (and deletion)
(...)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.1.1973-0f4abfbcc/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.15.3.876-ad6e39743/PLEX MEDIA SCANNER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.6.2368-97add474d/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.20.3.3437-f1f08d65b/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.7.2457-77cb9455c/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.20.1.3252-a78fef9a9/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.19.3.2764-ef515a800/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.5.2309-f5213a238/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.1.2019-c186313fe/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.15.8.1198-eadbcbb45/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.16.2.1321-ad17d5f9e/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.6.2368-97add474d/PLEX MEDIA SCANNER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.1.1973-0f4abfbcc/PLEX MEDIA SCANNER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.5.2309-f5213a238/PLEX MEDIA SCANNER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.20.1.3252-a78fef9a9/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.19.2.2737-b69929dab/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.19.5.3112-b23ab3896/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.16.5.1488-deeb86e7f/PLEX MEDIA SCANNER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.9.2578-513b381af/PLEX MEDIA SCANNER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.1.1973-0f4abfbcc/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.15.3.876-ad6e39743/PLEX DLNA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.16.2.1297-4b7ace214/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.19.3.2764-ef515a800/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.7.2457-77cb9455c/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.15.4.993-bb4a2cb6c/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.16.2.1321-ad17d5f9e/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.15.3.876-ad6e39743/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.16.2.1297-4b7ace214/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.20.3.3483-211702a9f/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.19.2.2737-b69929dab/PLEX MEDIA SCANNER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.4.2171-ac2afe5f8/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.9.2578-513b381af/PLEX MEDIA SERVER: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.18.9.2578-513b381af/PLEX TUNER SERVICE: Excluded from sync (and deletion)
2021/01/15 17:50:05 DEBUG : ssdpool/docker/plex/Library/Application Support/Plex Media Server/Crash Reports/1.19.1.2645-ccb6eb67e/PLEX MEDIA SCANNER: Excluded from sync (and deletion)

I'm also seeing this from the other excluded paths i.e

2021/01/15 17:50:06 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/vscode.workspaceFolders.js: Excluded from sync (and deletion)
2021/01/15 17:50:06 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/vscode.config.js: Excluded from sync (and deletion)
2021/01/15 17:50:06 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/validator.js: Excluded from sync (and deletion)
2021/01/15 17:50:06 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/util.js: Excluded from sync (and deletion)
2021/01/15 17:50:06 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/SuggestionsGenerator.js: Excluded from sync (and deletion)
2021/01/15 17:50:06 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/server.js: Excluded from sync (and deletion)
2021/01/15 17:50:06 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/package.json: Excluded from sync (and deletion)
2021/01/15 17:50:07 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/package-lock.json: Excluded from sync (and deletion)
2021/01/15 17:50:07 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/logger.js: Excluded from sync (and deletion)
2021/01/15 17:50:07 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/index.js: Excluded from sync (and deletion)
2021/01/15 17:50:07 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/documentSettings.js: Excluded from sync (and deletion)
2021/01/15 17:50:07 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/cspellConfig.js: Excluded from sync (and deletion)
2021/01/15 17:50:07 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/codeActions.js: Excluded from sync (and deletion)
2021/01/15 17:50:07 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/autoLoad.js: Excluded from sync (and deletion)
2021/01/15 17:50:07 DEBUG : datapool/docker2/codeserver/config/extensions/streetsidesoftware.code-spell-checker-1.8.0/server/api.js: Excluded from sync (and deletion)

I'm at a loss. Hopefully, it's a stupid typo somewhere (as it usually is).

Animosity022 · January 15, 2021, 5:36pm

What does a ls -al /data/host look like?

nle · January 15, 2021, 5:46pm

I'm guessing you mean ls /mnt/<host>

It lists two dirs:


drwxr-xr-x 11 nobody 4294967294   14 Dec 18 23:16 datapool
drwxr-xr-x  3 nobody 4294967294    4 Dec 18 22:51 ssdpool

Animosity022 · January 15, 2021, 6:00pm

Just for my own brain, is your folder actually called

<host>

with the <> in the name?

nle · January 15, 2021, 6:41pm

No , it’s an nfs share, so <host> is just a placeholder I used in this public forum for the actual hostname.

Animosity022 · January 15, 2021, 6:55pm

Hmm, I still don't see anything obvious as when I try to replicate, I don't get the same debug logs:

felix@flux:~$ rclone sync /home/felix/test GD:test --filter-from filter -vv
2021/01/15 13:55:17 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "sync" "/home/felix/test" "GD:test" "--filter-from" "filter" "-vv"]
2021/01/15 13:55:17 DEBUG : Creating backend with remote "/home/felix/test"
2021/01/15 13:55:17 DEBUG : Using config file from "/home/felix/.config/rclone/rclone.conf"
2021/01/15 13:55:17 DEBUG : Creating backend with remote "GD:test"
2021/01/15 13:55:19 DEBUG : ssdpool/docker/plex: Excluded
2021/01/15 13:55:19 DEBUG : ssdpool/hosts: Size and modification time the same (differ by 0s, within tolerance 1ms)
2021/01/15 13:55:19 DEBUG : ssdpool/hosts: Unchanged skipping
2021/01/15 13:55:19 DEBUG : Google drive root 'test': Waiting for checks to finish
2021/01/15 13:55:19 DEBUG : Google drive root 'test': Waiting for transfers to finish
2021/01/15 13:55:19 DEBUG : Waiting for deletions to finish
2021/01/15 13:55:19 INFO  : There was nothing to transfer
2021/01/15 13:55:19 INFO  :
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:         2.2s

2021/01/15 13:55:19 DEBUG : 5 go routines active
felix@flux:~$ rclone ls test/ssdpool/docker/plex/
        0 two/testtwo
        0 one/testone
felix@flux:~$

@ncw - any thoughts?

nle · January 15, 2021, 8:03pm

Yes, I'm at a loss. Appreciate the help though.

nle · January 18, 2021, 10:14am

Bumpetibump. 15 letters

system · March 20, 2021, 6:14am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.