Bisync Filtering not excluding folders

What is the problem you are having with rclone?

Hello Guys,
I have a Bisync connection between a local NAS and my Onedrive.
I only want to sync one folder (1_OnlineNAS) with my Onedrive. In here are some documents i want to back-up. All other folders i don't care.

Reading the filtering section this should be doable using - and + in a filter file. Just exclude everything, and only include the folder.
I tried two things. Relative paths :

   - /**
  + /1_OnlineNAS/

-> This gives an error since it then searches for the folder in the wrong location.

And absolute paths on my Pi :

-  /srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082/Files/**
+  /srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082/Files/1_OnlineNAS/**

-> This gives no error. But just syncs all folder, not only the one i need.

What i want to achieve is that only that one folder is synced. And if local or on the OneDrive other folders/files are made, they are all ignored.

Thanks for the help !

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.1
- os/version: raspbian 10.12
- os/kernel: 5.10.103-v7l+ (armv7l)
- os/type: linux
- os/arch: arm
- go/version: go1.18.5
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

OneDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

 sudo rclone --config=/home/pi/.config/rclone/rclone.conf bisync onedriveBjorn: /srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082/Files/  --filter-from /home/pi/FilterBjorn -P

The rclone config contents with secrets removed.

pi@pi:~ $ sudo rclone --config=/home/pi/.config/rclone/rclone.conf config
Current remotes:

Name                 Type
====                 ====
onedriveBjorn        onedrive
onedrive2        onedrive

A log from the command with the -vv flag

sudo rclone --config=/home/pi/.config/rclone/rclone.conf bisync onedriveBjorn: /srv/dev-disk-by-uuid-4bb55db4-aea0-4866                                                                                                            -876e-27fdc4bbf082/Files/  --filter-from /home/pi/FilterBjorn -P -vv
2022/08/21 09:50:48 DEBUG : rclone: Version "v1.59.1" starting with parameters ["rclone" "--config=/home/pi/.config/rclone/rclone                                                                                                            .conf" "bisync" "onedriveBjorn:" "/srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082/Files/" "--max-delete" "100" "--filt                                                                                                            er-from" "/home/pi/FilterBjorn" "-P" "-vv"]
2022/08/21 09:50:48 DEBUG : Creating backend with remote "onedriveBjorn:"
2022/08/21 09:50:48 DEBUG : Using config file from "/home/pi/.config/rclone/rclone.conf"
2022/08/21 09:50:49 DEBUG : Creating backend with remote "/srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082/Files/"
2022/08/21 09:50:49 NOTICE: bisync is EXPERIMENTAL. Don't use in production!
2022-08-21 09:50:49 DEBUG : Lock file created: /root/.cache/rclone/bisync/onedriveBjorn_..srv_dev-disk-by-uuid-4bb55db4-aea0-4866                                                                                                            -876e-27fdc4bbf082_Files.lck
2022-08-21 09:50:49 INFO  : Synching Path1 "onedriveBjorn:/" with Path2 "/srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf0                                                                                                            82/Files/"
2022-08-21 09:50:49 INFO  : Path1 checking for diffs
2022-08-21 09:50:51 INFO  : Path2 checking for diffs
2022-08-21 09:50:51 INFO  : No changes found
2022-08-21 09:50:51 INFO  : Updating listings
2022-08-21 09:50:51 INFO  : Validating listings for Path1 "onedriveBjorn:/" vs Path2 "/srv/dev-disk-by-uuid-4bb55db4-aea0-4866-87                                                                                                            6e-27fdc4bbf082/Files/"
2022-08-21 09:50:51 DEBUG : Lock file removed: /root/.cache/rclone/bisync/onedriveBjorn_..srv_dev-disk-by-uuid-4bb55db4-aea0-4866                                                                                                            -876e-27fdc4bbf082_Files.lck
2022-08-21 09:50:51 INFO  : Bisync successful
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:         2.6s
2022/08/21 09:50:51 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:         2.6s

2022/08/21 09:50:51 DEBUG : 5 go routines active

Paths are relative.

Can you use rclone ls for testing?

Filters are read top down so you want to flip them.

Run rclone ls with this and share the output.

  + /1_OnlineNAS/
  - /**
1 Like

Hello Animosity,
Thanks for the help.

I adjusted the filter document to relative paths, and put it in the same location as my folders to sync are.

I then first ran :

sudo rclone --config=/home/pi/.config/rclone/rclone.conf bisync onedriveBjorn: /srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082/Files/  --filter-from /srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082/Files/FilterBjorn.txt -P --resync
2022/08/21 14:40:26 NOTICE: bisync is EXPERIMENTAL. Don't use in production!
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:         2.2s

And after that : (removed --resync)

sudo rclone --config=/home/pi/.config/rclone/rclone.conf bisync onedriveBjorn: /srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082/Files/  --filter-from /srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082/Files/FilterBjorn.txt -P
2022/08/21 14:41:17 NOTICE: bisync is EXPERIMENTAL. Don't use in production!
2022-08-21 14:41:17 ERROR : Empty prior Path1 listing. Cannot sync to an empty directory: /root/.cache/rclone/bisync/onedriveBjorn_..srv_dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082_Files.path1.lst
2022-08-21 14:41:17 ERROR : Bisync critical error: empty prior Path1 listing: /root/.cache/rclone/bisync/onedriveBjorn_..srv_dev-disk-by-uuid-4bb55db4-aea0-4866-876e-27fdc4bbf082_Files.path1.lst
2022-08-21 14:41:17 ERROR : Bisync aborted. Must run --resync to recover.
Transferred:              0 B / 0 B, -, 0 B/s, ETA -

Which is the same error I got before. And was the reason i switched to absolute paths.
As you asked i also ran ls :

sudo rclone --config=/home/pi/.config/rclone/rclone.conf ls onedriveBjorn:
        3 1_OnlineNAS/BettyNas/test.txt
       26 1_OnlineNAS/BettyNas/testDocumentBetty.txt
        3 1_OnlineNAS/gr/test.txt
        0 1_OnlineNAS/BjornNas/Documents/TestDoc.txt
    16492 1_OnlineNAS/BjornNas/AutoMechanica/AutoOnderhoud.xlsx
    17865 1_OnlineNAS/BjornNas/AutoMechanica/Bestellijst.xlsx

For completness, here is ls-l on the file locations.
As you can see it did not make a FilterBjorn.txt.mk document like normal:

ls -l
total 20
drwxrwsrwx+ 4 root   users 4096 Aug 21 14:37 1_OnlineNAS
drwxrwsr-x+ 2 nobody users 4096 Aug 21 09:33 99_SystemLogs
-rwxrwxrwx+ 1 root   users   32 Aug 21 14:25 FilterBjorn.txt

Content of the filter file :

+ /1_OnlineNAS/
- /**

Thanks !

For completeness . I also tried again with absolute paths.
Then the command is accepted :

sudo rclone --config=/home/pi/.config/rclone/rclone.conf bisync onedriveBjorn: /srv/dev-disk-by-uuid-4bb55db4aea0-4866-876e-27fdc4bbf082/Files/  --filter-from /srv/dev-disk-by-uuid-4bb55db4-aea0-4866-876e27fdc4bbf082/Files/FilterBjorn.txt -P
2022/08/21 14:49:16 NOTICE: bisync is EXPERIMENTAL. Don't use in production!
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         5.6s

But the problem stays. If i make a new folder on my local drive its uploaded to the Onedrive.

Can we use rclone ls for testing?

felix@gemini:~/test$ rclone ls /home/felix/test --filter-from /home/felix/test/FilterBjorn.txt
      327 1_OnlineNAS/hosts

Need to add the ** to match all the files/directories if that's your goal.

felix@gemini:~/test$ cat /home/felix/test/FilterBjorn.txt
+ /1_OnlineNAS/**
- /**
felix@gemini:~/test$ rclone ls /home/felix/test --filter-from /home/felix/test/FilterBjorn.txt
      327 1_OnlineNAS/hosts
felix@gemini:~/test$ rclone ls /home/felix/test
       24 FilterBjorn.txt
        0 99_SystemLogs/two
      327 1_OnlineNAS/hosts

Please share the rclone ls output before doing anything else and validate it's doing what you want.

1 Like

The ** did the trick.
Stupid for me not to see it.

Anyway, thanks for the quick and useful help !
Really appreciate it.

1 Like

That's great news!

I missed it when I pasted the filter in as well so we did get there in the end.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.