Rclone - Transfer only a list of files from a specific folder

[Nuno_Bernardo]

Got it. Thanks by the information.
Do you know any way where I could automate and run a batch of like 100 rclone commands, so I could create a rclone command line for each file from a list? So I could use the --include command, per line, per each file?

[asdffdsa (jojothehumanmonkey)]

write a powershell script...

[calisro (Rob)]

What OS are you on? It should be easy to take a list of files and strip everything but the filename and prepend the filter criteria.

[albertony]

Given a file "includes.txt" with the following content:

file.txt
otherfile.txt
thirdfile.txt

You could execute the following in PowerShell:

Get-Content .\includes.txt | ForEach-Object { rclone <put_your_arguments_here> --include $_ }

Which gives same result as:

rclone <your_arguments_here> --include file.txt
rclone <your_arguments_here> --include otherfile.txt
rclone <your_arguments_here> --include thirdfile.txt

Or if you prefer Command Prompt:

for /F "tokens=*" %A in (includes.txt) do rclone <put_your_arguments_here> %A

(to put that in a batch file you must double up the % from %A to %%A)

[Nuno_Bernardo]

Hi,

Many thanks for your help. Your script worked as Intended.

I used:

Get-Content .\files-from.txt | ForEach-Object { .\rclone copy Wasabi:DemoFootage C:\Users\nmb\Desktop\Teste --include $_ }

On Windows Powershell.

There's only one error message with the --include parameter. It doesn't affect the results. See the screen below:

And it download the files and directories I wanted - listed on the .txt file, organized by folders like in the remote. It fixed the problem.

If it's not too much, what will be the one line script to use on Mac OSX Terminal? Will it be the same as the Powershell?

[albertony]

Does your file (files-from.txt) contain any empty lines?

If it does, you can skip them by inserting a where filter: | Where-Object { $_ } | ForEach-Object {...

Also, if you (temporarily) add -vv to the rclone command it will output debug information, including the actual arguments passed to it, which will help identify any issues.

If it's not too much, what will be the one line script to use on Mac OSX Terminal? Will it be the same as the Powershell?

You could install the cross platform PowerShell (aka. PowerShell Core) on osx and use the same command. See https://docs.microsoft.com/en-gb/powershell/scripting/install/installing-powershell-core-on-macos

Else, it depends on the native shell, if it is bash/-compatible you can do something like described here: https://stackoverflow.com/questions/1521462/looping-through-the-content-of-a-file-in-bash/1521498#1521498

[Nuno_Bernardo]

Thank you very much for all of your help. Yes, it had an empty line. Now I know.