Sync not deleting empty directories on Dropbox remote

What is the problem you are having with rclone?

When I run the command rclone sync /source remote:PATH/, files are removed correctly from the remote, but left-over empty directories are not removed.

If I run rclone tree remote:PATH it lists the empty directories.

This is happening with Dropbox as the remote.

Run the command 'rclone version' and share the full output of the command.

rclone v1.67.0
- os/version: unknown
- os/kernel: 6.10.0-rc6 (aarch64)
- os/type: linux
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.22.4
- go/linking: static
- go/tags: none

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
Yes

Which cloud storage system are you using? (eg Google Drive)

Dropbox

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync --progress --log-file /var/log/cloud_sync.log --filter-from /storage/.config/cloud_sync-rules.txt --delete-excluded --verbose /storage/roms dropbox:GAMES/

it also fails in the same way with the simpler

rclone sync -vv /storage/roms dropbox:GAMES/

The rclone config contents with secrets removed.

[dropbox]
type = dropbox
token = {***}

A log from the command with the -vv flag

2024/07/21 16:40:24 INFO  : savestates/snes/Super Bomberman (USA).state.auto: Deleted
2024/07/21 16:40:25 INFO  : savestates/snes/Super Bomberman (USA).state.auto.png: Deleted
2024/07/21 16:40:25 INFO  : savestates/nes/Yoshi (USA).state.auto.png: Deleted
2024/07/21 16:40:25 INFO  : savestates/gb/Alleyway (World).state.auto.png: Deleted
2024/07/21 16:40:25 INFO  : savestates/nes/Zelda II - The Adventure of Link (USA).state.auto: Deleted
2024/07/21 16:40:25 INFO  : savestates/gb/Alleyway (World).state.auto: Deleted
2024/07/21 16:40:25 INFO  : nes/Zelda II - The Adventure of Link (USA).srm: Deleted
2024/07/21 16:40:25 INFO  : savestates/nes/Yoshi (USA).state.auto: Deleted
2024/07/21 16:40:25 INFO  : savestates/nes/Zelda II - The Adventure of Link (USA).state.auto.png: Deleted
2024/07/21 16:40:26 INFO  : psx/Grand Theft Auto 2/Grand Theft Auto 2.srm: Deleted
2024/07/21 16:40:26 INFO  : savestates/snes: Removing directory
2024/07/21 16:40:27 INFO  : savestates/nes: Removing directory
2024/07/21 16:40:28 INFO  : savestates/gb: Removing directory
2024/07/21 16:40:29 INFO  : psx/Grand Theft Auto 2: Removing directory
2024/07/21 16:40:30 INFO  : There was nothing to transfer
2024/07/21 16:40:30 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                10 / 10, 100%
Deleted:               10 (files), 4 (dirs), 218.540 KiB (freed)
Elapsed time:         8.1s

2024/07/21 16:40:30 INFO  : Dropbox root 'GAMES': Committing uploads - please wait...

if I then run rclone tree dropbox:games I get:

/
|___ nes
|___ PSX
|___ savestates

which are all now empty directories.

I wonder why do you expect empty directories to be deleted?

When I sync from the remote to local, empty directories are removed.

Additionally, when I sync as specified, the parent directories that have had all of their files deleted are deleted as well.

The only directories not being deleted are these top-level directories in the tree.

Am I misreading the documentation with regards to how sync handles empty directories on the remote?

Sync should make destination the same as source. Maybe I missed something but it was never deleting already existing but empty dirs.

Exception is creating NEW empty dirs - it is off by default and has to be turned on using:

--create-empty-src-dirs   Create empty source dirs on destination after sync

Thanks so much. I think I have an odd edge case, and perhaps this is the expected behavior:

Step 1

Source:
/
|_Folder 1
|_Folder 2
|_Folder 3
|_Folder 4

Remote:
/
|_Folder 1
   |_File A
   |_File B
|_Folder 2
   |_Folder 5
      |_File C
|_Folder 3
   |_File D

Step 2: After running rclone sync

Source:
/
|_Folder 1
|_Folder 2
|_Folder 3
|_Folder 4

Remote:
/
|_Folder 1
|_Folder 2
|_Folder 3

So, a couple of things are happening:

  1. As expected, since I don't have the --create-empty-src-dirs flag, Folder 4 is not copied to the remote
  2. As expected, Folder 5, File A, File B, File C, and File D are all removed for the remote because they are not present on the source
  3. However, Folder 1, Folder 2, and Folder 3 remain on the remote after the sync. These directories are now empty, and I was expecting them to be removed.

Perhaps in the future there could be a --remove-empty-remote-dirs flag so that after running a sync like the above, the end result would be:

Source:
/
|_Folder 1
|_Folder 2
|_Folder 3
|_Folder 4

Remote:
/

The best thing to do is to share a debug log.

Example of an empty directory on a destination of a remote being deleted.

texter@Earls-Mac-mini test % rclone mkdir GD:two
texter@Earls-Mac-mini test % rclone mkdir GD:two/test
texter@Earls-Mac-mini test % rclone sync one GD:two -vv
2024/07/22 16:35:15 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "sync" "one" "GD:two" "-vv"]
2024/07/22 16:35:15 DEBUG : Creating backend with remote "one"
2024/07/22 16:35:15 DEBUG : Using config file from "/Users/texter/.config/rclone/rclone.conf"
2024/07/22 16:35:15 DEBUG : fs cache: renaming cache item "one" to be canonical "/Users/texter/Downloads/test/one"
2024/07/22 16:35:15 DEBUG : Creating backend with remote "GD:two"
2024/07/22 16:35:15 DEBUG : blah: Need to transfer - File not found at Destination
2024/07/22 16:35:16 DEBUG : Google drive root 'two': Waiting for checks to finish
2024/07/22 16:35:16 DEBUG : Google drive root 'two': Waiting for transfers to finish
2024/07/22 16:35:16 DEBUG : blah: md5 = d41d8cd98f00b204e9800998ecf8427e OK
2024/07/22 16:35:16 INFO  : blah: Copied (new)
2024/07/22 16:35:16 DEBUG : Waiting for deletions to finish
2024/07/22 16:35:16 INFO  : test: Removing directory
2024/07/22 16:35:17 DEBUG : Google drive root 'two': deleted 1 directories
2024/07/22 16:35:17 INFO  :
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Deleted:                0 (files), 1 (dirs)
Transferred:            1 / 1, 100%
Elapsed time:         1.7s

2024/07/22 16:35:17 DEBUG : 10 go routines active

Here are the results:

RK3566:~ # rclone sync /storage/Testing dropbox:Testing -vv
2024/07/22 17:28:03 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "sync" "/storage/Testing" "dropbox:Testing" "-vv"]
2024/07/22 17:28:03 DEBUG : Creating backend with remote "/storage/Testing"
2024/07/22 17:28:03 DEBUG : Using config file from "/storage/.config/rclone/rclone.conf"
2024/07/22 17:28:03 DEBUG : Creating backend with remote "dropbox:Testing"
2024/07/22 17:28:03 DEBUG : dropbox: Loaded invalid token from config file - ignoring
2024/07/22 17:28:03 DEBUG : Saving config "token" in section "dropbox" of the config file
2024/07/22 17:28:03 DEBUG : dropbox: Saved new token in config file
2024/07/22 17:28:05 DEBUG : Dropbox root 'Testing': Waiting for checks to finish
2024/07/22 17:28:05 DEBUG : Dropbox root 'Testing': Waiting for transfers to finish
2024/07/22 17:28:05 DEBUG : Waiting for deletions to finish
2024/07/22 17:28:05 INFO  : Folder 1/File A: Deleted
2024/07/22 17:28:06 INFO  : Folder 3/File D: Deleted
2024/07/22 17:28:06 INFO  : Folder 2/Folder 5/File C: Deleted
2024/07/22 17:28:06 INFO  : Folder 1/File B: Deleted
2024/07/22 17:28:06 INFO  : Folder 2/Folder 5: Removing directory
2024/07/22 17:28:07 DEBUG : Dropbox root 'Testing': deleted 1 directories
2024/07/22 17:28:07 INFO  : There was nothing to transfer
2024/07/22 17:28:07 INFO  : 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 4 / 4, 100%
Deleted:                4 (files), 1 (dirs), 0 B (freed)
Elapsed time:         3.7s

2024/07/22 17:28:07 DEBUG : 7 go routines active
2024/07/22 17:28:07 INFO  : Dropbox root 'Testing': Committing uploads - please wait...
RK3566:~ # rclone tree dropbox:Testing
/
├── Folder 1
├── Folder 2
└── Folder 3

0 directories, 0 files
RK3566:~ # ls /storage/Testing
Folder 1  Folder 2  Folder 3  Folder 4

Guessing Folder 4 is empty? If yes, everything is operating exactly as expected as your

Non matching stuff was removed. By default, you don't create empty source directories on the remote.

Yes, sorry, I should have said that the sync matched the structure I game in my example.

The unexpected behavior for me is just that the matching but empty directories remain on the remote. If it was a new sync, they wouldn't be transferred, and so I was expecting the end result of the test I did to be the same as if the initial root directory was empty, which would result in it remaining empty.

I may be the only one who would want a flag like --remove-empty-remote-dirs (or maybe it would be called --remove-empty-src-dirs), but that would help keep it clean on the remote when not using --create-empty-src-dirs.

Let me see I can rephrase that.

I have a source directory.
The source directory is empty.
The destination already has that same source directory.

If that destination directory is already empty, rclone does nothing.
If that destination directory has anything, it'll be deleted but the directory remains.

So the source and destination match.

texter@Earls-Mac-mini test % rclone copy /etc/hosts GD:two/emptydir
texter@Earls-Mac-mini test % rclone sync one GD:two -vv
2024/07/22 19:52:39 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "sync" "one" "GD:two" "-vv"]
2024/07/22 19:52:39 DEBUG : Creating backend with remote "one"
2024/07/22 19:52:39 DEBUG : Using config file from "/Users/texter/.config/rclone/rclone.conf"
2024/07/22 19:52:39 DEBUG : fs cache: renaming cache item "one" to be canonical "/Users/texter/Downloads/test/one"
2024/07/22 19:52:39 DEBUG : Creating backend with remote "GD:two"
2024/07/22 19:52:40 DEBUG : emptydir: Directory modification time the same (differ by -416.48µs, within tolerance 1ms)
2024/07/22 19:52:40 DEBUG : blah: Size and modification time the same (differ by -743.147µs, within tolerance 1ms)
2024/07/22 19:52:40 DEBUG : blah: Unchanged skipping
2024/07/22 19:52:40 DEBUG : Google drive root 'two': Waiting for checks to finish
2024/07/22 19:52:40 DEBUG : Google drive root 'two': Waiting for transfers to finish
2024/07/22 19:52:40 DEBUG : Waiting for deletions to finish
2024/07/22 19:52:40 INFO  : emptydir/hosts: Deleted
2024/07/22 19:52:40 INFO  : There was nothing to transfer
2024/07/22 19:52:40 INFO  :
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 2 / 2, 100%
Deleted:                1 (files), 0 (dirs)
Elapsed time:         0.8s

2024/07/22 19:52:40 DEBUG : 7 go routines active

and

texter@Earls-Mac-mini test % rclone lsf GD:two
blah
emptydir/

So you want something brand new that says if source directory is empty, delete the destination directory.

Not sure the use case for it, but feel free to put in a feature request.

Will do! The use case is limited, but related to some open source emulation projects where there are source directories for possible systems, with existing folders for each system upon install of the OS. The user then puts the correct files into the corresponding directory. This ensures that the directory structure of the source that files are being sorted into is consistent for all users.

When backing up files to a remote via rclone sync, it can leave orphaned directories on the remote. These empty directories can build up over time, and it forces the user to have empty directories in their backup. The issue for the user is that they have additional directories to traverse when looking for something and general clutter.

Ideally, if the states change to a point where a non-empty source directory has been copied to the remote and then the source directory later becomes empty, the remote directory would be deleted as well. This would ensure that this flow matches what would happen if a source directory is empty when running rclone sync without the --create-empty-src-dirs flag so that an empty source directory is not present on the the remote.

Makes sense. Text reads a little bit. Each person's use case is valuable and highly important to them as I just meant I couldn't think of one for me :slight_smile:

I personally expect sync to sync 100% of the time and exactly match as that's what I am used to so back in the day when it didn't have a flag to create empty directories, that was strange to me.

Thanks for all of the help! I've filed a feature request.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.