Remove all second+ level directories, leave first level

What is the problem you are having with rclone?

I'd like delete all empty subdirectories excluding the first level. This would be part of a larger move operation, but I'd be happy to make a second pass with rmdirs too if necessary.

Imagine this structure:

/
├── 1
│   ├── a
│   │   └── test.txt
│   ├── b
│   │   └── test.txt
│   └── test.txt
├── 2
│   └── test.txt
└── file.txt

What I want to remain is just:

/
├── 1
├── 2

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.0

  • os/version: Microsoft Windows 11 Pro 22H2 (64 bit)
  • os/kernel: 10.0.22622.575 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.18.3
  • go/linking: static
  • go/tags: cmount
    -->

The command you were trying to run

I can include just the directories I want to keep:

--filter "+ /*/" --filter "- **"

But I can't flip it around to exclude those without also preventing rclone from recursing into them.

rclone rmdirs . --filter "+ /*/**" --filter "- /*/"

I tried various regex forms, again I can select just what I want to keep, but I can't flip it around to negate just those without excluding everything within. I was hoping that by using regex to block directory filter rules I could force it to get a full list first which would give me a chance to parse later, but I can't manage it.

I suspect I'm just hitting a design limit of rclone as it needs to include a path to be willing to recurse into the path, which seems pretty reasonable.

I could script it, output a list of the first level directories (that I want to keep) and rmdirs each of them individually, but I was hoping there was as cleaner way to do this within one command.

I've read this a few times and not sure I can think of a one liner to tackle it.

Rclone filtering is file based so doing directory level filtering won't really work.

If it was me, I'd just do the latter and script it out. If this is more a one time thing, it'll be less painful than trying to devise a one liner imo.

Perhaps a smarter person will think of something clever though :slight_smile:

For a one off I would have whipped together a quick script to do it in less time than I spent testing and writing the post :smiley: But it is for an ongoing project, with the first level of directories not being under my control so I need to make sure I've fully wrapped my head around maliciously named directories causing the shell to do uncomfortable things.

At this point I'm thinking I'll run the rclone move task, if it moved any files than call a script to pull the first level directories and kick off a bunch of rclone rmdirs, one for each subdirectory. Or possibly rethink a bit to see if I can lock down the list of first level directories.

Thanks!

I also cannot make an rclone one liner, but here is my best shot at a (reasonable) robust PowerShell script to delete all the empty subdirectories excluding the first level:

# Making Powershell UTF-8 aware: https://stackoverflow.com/questions/49476326/displaying-unicode-in-powershell/49481797#49481797
$OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding

$remote = "OneDrive:testfolder"
$dirs = $(./rclone lsjson $remote --dirs-only | ConvertFrom-Json)
foreach ($dir in $dirs) {
    ./rclone rmdirs --dry-run --leave-root $($remote + "/" + $dir.Name) -v
}

I have assumed that you have moved some/all of the test.txt somewhere else before executing this. If you want it to simply delete all your test.txt files then just replace rmdirs with something like this (not tested!):

./rclone purge --dry-run $($remote + "/" + $dir.Name) -v
./rclone mkdir $($remote + "/" + $dir.Name) -v

If you want the script to purge the entire structure and recreate the toplevel folders then it could look something like this (not tested!):

# Making Powershell UTF-8 aware: https://stackoverflow.com/questions/49476326/displaying-unicode-in-powershell/49481797#49481797
$OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding

$remote = "OneDrive:testfolder"
$dirs = $(./rclone lsjson $remote --dirs-only | ConvertFrom-Json)
./rclone purge --dry-run $remote -v
foreach ($dir in $dirs) {
    ./rclone mkdir --dry-run $($remote + "/" + $dir.Name) -v
}

Wait a second, how about this:

rclone rmdirs remote: --filter="+ /*/*/**" --filter="- /*/" --filter="+ /*/**"

Not fully tested, but might do it :sweat_smile:

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.