Move and Delete script for unRAID

What is the problem you are having with rclone?

I am not having a problem, I would just like some help :slight_smile:

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.1
- os/version: slackware 14.2+ (64 bit)
- os/kernel: 5.10.28-Unraid (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.9
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

SFTP

The command you were trying to run (eg rclone copy /tmp remote:tmp)

function DownloadFolders($RemoteRoot, $LocalRoot) {
    rclone lsf --dirs-only --dir-slash=false $RemoteRoot | ForEach-Object {
        $LocalPath = Join-Path -Path $LocalRoot -ChildPath $_
        Remove-Item -Recurse -Force -LiteralPath $LocalPath -ErrorAction Ignore
        rclone move --progress --transfers=1 "${RemoteRoot}/${_}" $LocalPath
    }
    rclone rmdirs $RemoteRoot --leave-root
}
DownloadFolders "ftpserver:test/" "I:\"
DownloadFolders "ftpserver:Another Folder/" "E:\Another Folder"

The rclone config contents with secrets removed.

[ftpserver]
type = sftp
host = HOST HERE
user = USERNAME HERE
pass = PASSWORD HERE
md5sum_command = md5sum
sha1sum_command = sha1sum

A log from the command with the -vv flag

I haven't actually run the command as it's for Powershell instead of Linux.

Reason for my post

I have been using the above script for a while now and it's been working great (thanks for everyone's help with it), however, I have just built myself a NAS that uses unRAID and I would like to have the same script that I use on Windows to work on unRAID. How can it be converted to work on unRAID?

The original post is here - Move and Delete

Thanks

I have been trying to find someone on Fiverr to do this script but what does the scirpt need to be written in? Shell?

Would this work? I took what @ncw wrote in my Move and Delete post and edited it a bit.

for folder in $(rclone lsf --dirs-only "ftpserver:Test Folder"); do
    rm -rf "/mnt/user/Media/Test Folder/${folder}"
    rclone move "ftpserver:Test Folder/${folder}" "/mnt/user/Media/Test Folder/${folder}" --progress --transfers=1 --dry-run
done

You can run powershell scripts on linux:

Thanks Nick. I wasn't aware of that, I'm new to Linux and unRAID. I wouldn't know where to start or how to install it on unRAID.

Would I be better using a script that is written specifically for Linux like the one you wrote before that I edited above? I'm scared to run it as not sure if it will delete everything.

I bit the bullet and tested the following script:

for folder in $(rclone lsf --dirs-only "ftpserver:Test Folder"); do
    echo rm -rf "/mnt/user/Media/Test Folder/${folder}"
    rclone move "ftpserver:Test Folder/${folder}" "/mnt/user/Media/Test Folder/${folder}" --progress --transfers=1 --dry-run
done

I don't think it worked as intended... This is what I got when running it using the User Scripts plugin:

Script location: /tmp/user.scripts/tmpScripts/Test/script
Note that closing this window will abort the execution of this script
rm -rf /mnt/user/Media/Test Folder/Folder

2022-06-03 16:46:21 ERROR : sftp://username@serveraddress.com:22/Test Folder/Folder: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:21 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:21 ERROR : sftp://username@serveraddress.com:22/Test Folder/Folder: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:21 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:21 ERROR : sftp://username@serveraddress.com:22/Test Folder/Folder: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:21 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2sTransferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022/06/03 16:46:21 NOTICE:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s

2022/06/03 16:46:21 Failed to move: directory not found
rm -rf /mnt/user/Media/Test Folder/1/

2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/1/: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/1/: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/1/: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2sTransferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022/06/03 16:46:22 NOTICE:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s

2022/06/03 16:46:22 Failed to move: directory not found
rm -rf /mnt/user/Media/Test Folder/Folder

2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/Folder: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/Folder: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/Folder: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2sTransferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022/06/03 16:46:22 NOTICE:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s

2022/06/03 16:46:22 Failed to move: directory not found
rm -rf /mnt/user/Media/Test Folder/2/

2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/2/: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/2/: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/2/: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2sTransferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022/06/03 16:46:22 NOTICE:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s

2022/06/03 16:46:22 Failed to move: directory not found
rm -rf /mnt/user/Media/Test Folder/Folder

2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/Folder: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/Folder: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : sftp://username@serveraddress.com:22/Test Folder/Folder: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:22 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2sTransferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022/06/03 16:46:22 NOTICE:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s

2022/06/03 16:46:22 Failed to move: directory not found
rm -rf /mnt/user/Media/Test Folder/3/

2022-06-03 16:46:23 ERROR : sftp://username@serveraddress.com:22/Test Folder/3/: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:23 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:23 ERROR : sftp://username@serveraddress.com:22/Test Folder/3/: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:23 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022-06-03 16:46:23 ERROR : sftp://username@serveraddress.com:22/Test Folder/3/: error reading source root directory: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.2s
2022-06-03 16:46:23 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2sTransferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s
2022/06/03 16:46:23 NOTICE:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.2s

2022/06/03 16:46:23 Failed to move: directory not found

The structure inside the Test Folder is as follows:

Folder 1
- file_example_MP3_1MG.mp3

Folder 2
- file_example_MP3_2MG.mp3

Folder 3
 - file_example_MP3_5MG.mp3

Sorry for so many posts, I'm just eager to get this working and trying to understand/learn what is going wrong/how to get it to work.

From my research it looks like the for loop doesn't like the use of ls or in this case lsf. So, I'm not sure how to get around that.

Your script looks pretty good.

Try putting echo in front of the rclone statements too and see if you can spot what is going wrong

for folder in $(rclone lsf --dirs-only "ftpserver:Test Folder"); do
    echo rm -rf "/mnt/user/Media/Test Folder/${folder}"
    echo rclone move "ftpserver:Test Folder/${folder}" "/mnt/user/Media/Test Folder/${folder}" --progress --transfers=1 --dry-run
done

Thanks Nick.

To me it looks like the folders are being split up after a space. So I currently have these folders on the remote:

Test 1/
Test 2/
Test 3/

And the output of the script is the following:

Script location: /tmp/user.scripts/tmpScripts/Test/script
Note that closing this window will abort the execution of this script
rm -rf /mnt/user/Media/Test Folder/Test
rclone move ftpserver:Test Folder/Test /mnt/user/Media/Test Folder/Test --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/1/
rclone move ftpserver:Test Folder/1/ /mnt/user/Media/Test Folder/1/ --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/Test
rclone move ftpserver:Test Folder/Test /mnt/user/Media/Test Folder/Test --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/2/
rclone move ftpserver:Test Folder/2/ /mnt/user/Media/Test Folder/2/ --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/Test
rclone move ftpserver:Test Folder/Test /mnt/user/Media/Test Folder/Test --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/3/
rclone move ftpserver:Test Folder/3/ /mnt/user/Media/Test Folder/3/ --progress --transfers=1 --dry-run

It thinks that there's the following folders:

Test
1
2
3

I have tried what is suggested here but it hasn't worked - https://www.cyberciti.biz/tips/handling-filenames-with-spaces-in-bash.html

Just taking a shot in the dark here, but in your script try putting your variable in double quotes.

rm -rf "/mnt/user/Media/Test Folder/"${folder}""

Thanks, I just tried that and it's still splitting the folders.

Script location: /tmp/user.scripts/tmpScripts/Test/script
Note that closing this window will abort the execution of this script
rm -rf /mnt/user/Media/Test Folder/Test
rclone move ftpserver:Test Folder/Test /mnt/user/Media/Test Folder/Test --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/1/
rclone move ftpserver:Test Folder/1/ /mnt/user/Media/Test Folder/1/ --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/Test
rclone move ftpserver:Test Folder/Test /mnt/user/Media/Test Folder/Test --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/2/
rclone move ftpserver:Test Folder/2/ /mnt/user/Media/Test Folder/2/ --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/Test
rclone move ftpserver:Test Folder/Test /mnt/user/Media/Test Folder/Test --progress --transfers=1 --dry-run
rm -rf /mnt/user/Media/Test Folder/3/
rclone move ftpserver:Test Folder/3/ /mnt/user/Media/Test Folder/3/ --progress --transfers=1 --dry-run

Using while read loop instead is the solution, I think, see this for inspiration: RClone / Bash - Loop through and download all files in directory individually - Stack Overflow

Edit: Maybe like this:

while read -r folder; do
    echo rm -rf "/mnt/user/Media/Test Folder/${folder}"
    echo rclone move "ftpserver:Test Folder/${folder}" "/mnt/user/Media/Test Folder/${folder}" --progress --transfers=1 --dry-run
done < <(rclone lsf --dirs-only "ftpserver:Test Folder")

Thank you. I think that's very close. It's leaving the folders on the remote and then when it's run again it then deletes the folders on local.

Would I add rclone rmdirs --leave-root onto the end?

What do you mean? Does it move the content but leaves the "ftpserver:Test Folder/${folder}" behind as empty folder or what? Running with -vv might give some relevant hints.

That's right it leaves the empty folders behind.

Below is the output when running -vv

Script location: /tmp/user.scripts/tmpScripts/Test/script
Note that closing this window will abort the execution of this script
2022/06/05 18:33:42 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "move" "ftpserver:Test Folder/Test 1/" "/mnt/user/Media/Test Folder/Test 1/" "--progress" "--transfers=1" "-vv"]
2022/06/05 18:33:42 DEBUG : Creating backend with remote "ftpserver:Test Folder/Test 1/"
2022/06/05 18:33:42 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf"
2022/06/05 18:33:42 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 1/: New connection 192.168.0.51:38142->185.21.216.160:22 to "SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u7"
2022/06/05 18:33:42 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 1/: Using absolute root directory "/media/sdh1/matt3/Test Folder/Test 1"
2022/06/05 18:33:42 DEBUG : Creating backend with remote "/mnt/user/Media/Test Folder/Test 1/"

2022-06-05 18:33:42 DEBUG : Local file system at /mnt/user/Media/Test Folder/Test 1/: Waiting for checks to finish
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.2s
Checking:

Transferring:
* file_example_MP3_5MG.mp3: transferring
2022-06-05 18:33:42 DEBUG : Local file system at /mnt/user/Media/Test Folder/Test 1/: Waiting for transfers to finish
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.2s
Checking:

Transferring:
* file_example_MP3_5MG.mp3: transferringTransferred: 2.996 MiB / 5.044 MiB, 59%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.7s
Checking:

Transferring:
* file_example_MP3_5MG.mp3: 59% /5.044Mi, 0/s, -
2022-06-05 18:33:43 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 1/: Running remote command: md5sum /media/sdh1/matt3/Test\ Folder/Test\ 1/file_example_MP3_5MG.mp3
Transferred: 5.044 MiB / 5.044 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.9s
Checking:

Transferring:
* file_example_MP3_5MG.mp3:100% /5.044Mi, 0/s, -
2022-06-05 18:33:43 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 1/: Remote command result: 2688ed914ed6315229fb89e1e916cdee /media/sdh1/matt3/Test Folder/Test 1/file_example_MP3_5MG.mp3
Transferred: 5.044 MiB / 5.044 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.9s
Checking:

Transferring:
* file_example_MP3_5MG.mp3:100% /5.044Mi, 0/s, -
2022-06-05 18:33:43 DEBUG : file_example_MP3_5MG.mp3: md5 = 2688ed914ed6315229fb89e1e916cdee OK
Transferred: 5.044 MiB / 5.044 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.9s
Checking:

Transferring:
* file_example_MP3_5MG.mp3:100% /5.044Mi, 0/s, -
2022-06-05 18:33:43 INFO : file_example_MP3_5MG.mp3: Copied (new)
Transferred: 5.044 MiB / 5.044 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.9s
Checking:

Transferring:
* file_example_MP3_5MG.mp3:100% /5.044Mi, 0/s, -
2022-06-05 18:33:43 INFO : file_example_MP3_5MG.mp3: Deleted
Transferred: 5.044 MiB / 5.044 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Deleted: 1 (files), 0 (dirs)
Transferred: 1 / 1, 100%
Elapsed time: 0.9s
Checking:
* file_example_MP3_5MG.mp3: checkingTransferred: 5.044 MiB / 5.044 MiB, 100%, 0 B/s, ETA -
Checks: 2 / 2, 100%
Deleted: 1 (files), 0 (dirs)
Renamed: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.9s
2022/06/05 18:33:43 INFO :
Transferred: 5.044 MiB / 5.044 MiB, 100%, 0 B/s, ETA -
Checks: 2 / 2, 100%
Deleted: 1 (files), 0 (dirs)
Renamed: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.9s

2022/06/05 18:33:43 DEBUG : 13 go routines active
2022/06/05 18:33:43 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "move" "ftpserver:Test Folder/Test 2/" "/mnt/user/Media/Test Folder/Test 2/" "--progress" "--transfers=1" "-vv"]
2022/06/05 18:33:43 DEBUG : Creating backend with remote "ftpserver:Test Folder/Test 2/"
2022/06/05 18:33:43 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf"
2022/06/05 18:33:43 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 2/: New connection 192.168.0.51:38144->185.21.216.160:22 to "SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u7"
2022/06/05 18:33:43 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 2/: Using absolute root directory "/media/sdh1/matt3/Test Folder/Test 2"
2022/06/05 18:33:43 DEBUG : Creating backend with remote "/mnt/user/Media/Test Folder/Test 2/"

2022-06-05 18:33:43 DEBUG : Local file system at /mnt/user/Media/Test Folder/Test 2/: Waiting for checks to finish
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.2s
Checking:

Transferring:
* file_example_MP3_2MG.mp3: transferring
2022-06-05 18:33:43 DEBUG : Local file system at /mnt/user/Media/Test Folder/Test 2/: Waiting for transfers to finish
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.2s
Checking:

Transferring:
* file_example_MP3_2MG.mp3: transferring
2022-06-05 18:33:43 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 2/: Running remote command: md5sum /media/sdh1/matt3/Test\ Folder/Test\ 2/file_example_MP3_2MG.mp3
Transferred: 2.016 MiB / 2.016 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.6s
Checking:

Transferring:
* file_example_MP3_2MG.mp3:100% /2.016Mi, 0/s, -
2022-06-05 18:33:43 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 2/: Remote command result: 0e64629c86cf0b24262122bf2a6eee23 /media/sdh1/matt3/Test Folder/Test 2/file_example_MP3_2MG.mp3
Transferred: 2.016 MiB / 2.016 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.6s
Checking:

Transferring:
* file_example_MP3_2MG.mp3:100% /2.016Mi, 0/s, -
2022-06-05 18:33:43 DEBUG : file_example_MP3_2MG.mp3: md5 = 0e64629c86cf0b24262122bf2a6eee23 OK
Transferred: 2.016 MiB / 2.016 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.6s
Checking:

Transferring:
* file_example_MP3_2MG.mp3:100% /2.016Mi, 0/s, -
2022-06-05 18:33:43 INFO : file_example_MP3_2MG.mp3: Copied (new)
Transferred: 2.016 MiB / 2.016 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.6s
Checking:

Transferring:
* file_example_MP3_2MG.mp3:100% /2.016Mi, 0/s, -
2022-06-05 18:33:43 INFO : file_example_MP3_2MG.mp3: Deleted
Transferred: 2.016 MiB / 2.016 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Deleted: 1 (files), 0 (dirs)
Transferred: 1 / 1, 100%
Elapsed time: 0.6s
Checking:
* file_example_MP3_2MG.mp3: checkingTransferred: 2.016 MiB / 2.016 MiB, 100%, 0 B/s, ETA -
Checks: 2 / 2, 100%
Deleted: 1 (files), 0 (dirs)
Renamed: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.6s
2022/06/05 18:33:43 INFO :
Transferred: 2.016 MiB / 2.016 MiB, 100%, 0 B/s, ETA -
Checks: 2 / 2, 100%
Deleted: 1 (files), 0 (dirs)
Renamed: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.6s

2022/06/05 18:33:43 DEBUG : 13 go routines active
2022/06/05 18:33:43 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rcloneorig" "--config" "/boot/config/plugins/rclone/.rclone.conf" "move" "ftpserver:Test Folder/Test 3/" "/mnt/user/Media/Test Folder/Test 3/" "--progress" "--transfers=1" "-vv"]
2022/06/05 18:33:43 DEBUG : Creating backend with remote "ftpserver:Test Folder/Test 3/"
2022/06/05 18:33:43 DEBUG : Using config file from "/boot/config/plugins/rclone/.rclone.conf"
2022/06/05 18:33:43 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 3/: New connection 192.168.0.51:38146->185.21.216.160:22 to "SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u7"
2022/06/05 18:33:44 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 3/: Using absolute root directory "/media/sdh1/matt3/Test Folder/Test 3"
2022/06/05 18:33:44 DEBUG : Creating backend with remote "/mnt/user/Media/Test Folder/Test 3/"

2022-06-05 18:33:44 DEBUG : Local file system at /mnt/user/Media/Test Folder/Test 3/: Waiting for checks to finish
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.2s
Checking:

Transferring:
* file_example_MP3_1MG.mp3: transferring
2022-06-05 18:33:44 DEBUG : Local file system at /mnt/user/Media/Test Folder/Test 3/: Waiting for transfers to finish
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.2s
Checking:

Transferring:
* file_example_MP3_1MG.mp3: transferring
2022-06-05 18:33:44 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 3/: Running remote command: md5sum /media/sdh1/matt3/Test\ Folder/Test\ 3/file_example_MP3_1MG.mp3
Transferred: 1.037 MiB / 1.037 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.4s
Checking:

Transferring:
* file_example_MP3_1MG.mp3:100% /1.037Mi, 0/s, -
2022-06-05 18:33:44 DEBUG : sftp://username@serveraddress.com:22/Test Folder/Test 3/: Remote command result: ae9120835f658f1ae57e5754811a9475 /media/sdh1/matt3/Test Folder/Test 3/file_example_MP3_1MG.mp3
Transferred: 1.037 MiB / 1.037 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.4s
Checking:

Transferring:
* file_example_MP3_1MG.mp3:100% /1.037Mi, 0/s, -
2022-06-05 18:33:44 DEBUG : file_example_MP3_1MG.mp3: md5 = ae9120835f658f1ae57e5754811a9475 OK
Transferred: 1.037 MiB / 1.037 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.4s
Checking:

Transferring:
* file_example_MP3_1MG.mp3:100% /1.037Mi, 0/s, -
2022-06-05 18:33:44 INFO : file_example_MP3_1MG.mp3: Copied (new)
Transferred: 1.037 MiB / 1.037 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Transferred: 0 / 1, 0%
Elapsed time: 0.4s
Checking:

Transferring:
* file_example_MP3_1MG.mp3:100% /1.037Mi, 0/s, -
2022-06-05 18:33:44 INFO : file_example_MP3_1MG.mp3: Deleted
Transferred: 1.037 MiB / 1.037 MiB, 100%, 0 B/s, ETA -
Checks: 0 / 1, 0%
Deleted: 1 (files), 0 (dirs)
Transferred: 1 / 1, 100%
Elapsed time: 0.4s
Checking:
* file_example_MP3_1MG.mp3: checkingTransferred: 1.037 MiB / 1.037 MiB, 100%, 0 B/s, ETA -
Checks: 2 / 2, 100%
Deleted: 1 (files), 0 (dirs)
Renamed: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.4s
2022/06/05 18:33:44 INFO :
Transferred: 1.037 MiB / 1.037 MiB, 100%, 0 B/s, ETA -
Checks: 2 / 2, 100%
Deleted: 1 (files), 0 (dirs)
Renamed: 1
Transferred: 1 / 1, 100%
Elapsed time: 0.4s

2022/06/05 18:33:44 DEBUG : 12 go routines active

I think you were correct with this. Either rclone rmdirs "ftpserver:Test Folder/${folder}" within (at end of) the loop, or rclone rmdirs --leave-root "ftpserver:Test Folder" after the loop.

1 Like

I tried rclone rmdirs --leave-root "ftpserver:Test Folder" at the end of the loop but that didn't seem to work.

However, the below looks like it's working after a few tests. I'll continue to test some more and fingers crossed this seems to be the one.

while read -r folder; do
    rm -rf "/mnt/user/Media/Test Folder/${folder}"
    rclone move "ftpserver:Test Folder/${folder}" "/mnt/user/Media/Test Folder/${folder}" --progress --transfers=1 -vv
    rclone rmdirs "ftpserver:Test Folder/${folder}"
done < <(rclone lsf --dirs-only "ftpserver:Test Folder")

Thanks so much for your help. I tried to find another way to change the for loop before you suggested the while read loop but never came across the while read loop.

After some more testing it looks like the above script works perfectly :slight_smile:

Thanks everyone for the help. I really appreciate it.

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.