Rclone cannot move multiple files at once

Say I have a couple of files I wish to move to the files folder on the remote.
rclone move *.mkv <remote>:files doesn't work because it expects only one item (why?).

Workaround: create a new folder, move the files there, then move the folder to the remote.

In that case, the files are only moved from the folder to the remote, not the actual folder (which should be moved too, IMO).

A command like this, where I want to move a folder to the root of the remote should be:

rclone move folder1 <remote>:

instead of

rclone move folder1 <remote>:folder1.

Not sure why rclone behaves like this.

Can you give an example of what you mean by using a few files?

If you wanted to move 2 files into a remote, you'd just include the path to them.

felix@gemini:~/test$ ls
one  two

rclone move /home/felix/test GD:

felix@gemini:~/test$ rclone lsf GD:
one
two

or are you trying to do

felix@gemini:~/test$ cd
felix@gemini:~$ pwd
/home/felix
felix@gemini:~$ rclone move /home/felix/test GD:test
felix@gemini:~$ cd test
felix@gemini:~/test$ ls
felix@gemini:~/test$ cd ..
felix@gemini:~$ rclone ls GD:test
      248 two
      248 one

You can use either of these to clean up either way too:

      --create-empty-src-dirs   Create empty source dirs on destination after move
      --delete-empty-src-dirs   Delete empty source dirs after move

I want to move 2 files from my home folder, not everything in my home folder.

This was the error I received:

agneev@do1:~$ rclone move *.mkv media:drop --dry-run 
Usage:
  rclone move source:path dest:path [flags]

Flags:
      --create-empty-src-dirs   Create empty source dirs on destination after move
      --delete-empty-src-dirs   Delete empty source dirs after move
  -h, --help                    help for move

Use "rclone [command] --help" for more information about a command.
Use "rclone help flags" for to see the global flags.
Use "rclone help backends" for a list of supported services.
Command move needs 2 arguments maximum: you provided 5 non flag arguments: ["jellyfish-25-mbps-hd-h264.mkv" "jellyfish-40-mbps-hd-hevc-10bit.mkv" "jellyfish-50-mbps-hd-h264.mkv" "jellyfish-50-mbps-hd-hevc.mkv" "media:drop"]

I'm not sure I can explain it any better.

If I use mv ~/folder1 /mnt/<rclone_mount>, the folder and all contents beneath it are moved.

whereas using rclone move ~/folder1 <remote>: moves the contents of folder1 but not folder1 itself. It should behave similar to mv right?

Not sure why those flags exist at all.

Why would the folders remain if move was used?

Not sure what this one does.

You can't use wildcards on those commands, you'd have to use a filter.

You use the filter or use two commands to do the same thing.

felix@gemini:~/test$ rclone move one GD:
rclone movfelix@gemini:~/test$ rclone move two GD: -vv
2020/03/10 05:50:12 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "move" "two" "GD:" "-vv"]
2020/03/10 05:50:12 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2020/03/10 05:50:12 DEBUG : two: Need to transfer - File not found at Destination
2020/03/10 05:50:13 DEBUG : two: MD5 = 6d6862970272958c532ec4af65af8386 OK
2020/03/10 05:50:13 INFO  : two: Copied (new)
2020/03/10 05:50:13 INFO  : two: Deleted
2020/03/10 05:50:13 INFO  :
Transferred:   	       248 / 248 Bytes, 100%, 199 Bytes/s, ETA 0s
Checks:                 2 / 2, 100%
Deleted:                1
Transferred:            1 / 1, 100%
Elapsed time:         1.2s

2020/03/10 05:50:13 DEBUG : 5 go routines active
2020/03/10 05:50:13 DEBUG : rclone: Version "v1.51.0" finishing with parameters ["rclone" "move" "two" "GD:" "-vv"]

What's the actual final output look like for you?

I have

/home/felix/test and it contains a file one and two

felix@gemini:~/test$ ls
one  two

If the remote is GD:, what does it look like on the other side?

GD:test and that contains one and two
or
GD: and that contains one and two?

You're moving two folders, one after the other, so of course it will work.

The bottom line is this needs to be rectified so that it works similar to mv.

@ncw

No, I'm moving two files, one a time.

felix@gemini:~/test$ rclone lsf GD:test
one
two

If you can help me get your use case, it's an easy answer.

It doesn't work like mv in terms of wild carding and that won't change most likely, which is why filters exist. I'm happy to help if you want it.

I will take you up on that. I want to move all folders under, say, media:drop such that there's no leftovers; everything's moved over.

Even better, a way to find all files ending with mkv under the subfolders of drop and move them (and possibly delete the folders from which the mkv files are being moved?? I know its a stretch and its perfectly fine if that can't be done).

I tried making a script, not sure whether it'll work the way its supposed to.

#!/bin/bash
nohup /usr/bin/rclone move media:drop /mnt/1TB/drop --multi-thread-streams 4 -P --transfers=6 --delete-empty-src-dirs > /tmp/move.log &

So like exactly my use case I do every night? :slight_smile:

I have a folder called /local

It contains a structure like:

felix@gemini:/local$ ls
Movies  NZB  seed  TV

I want to move the whole folder Movies and TV every night but I leave NZB and seed so I use an exclude file with this command:

/usr/bin/rclone move /local/ gcrypt: --log-file /opt/rclone/logs/upload.log -v --exclude-from /opt/rclone/scripts/excludes --delete-empty-src-dirs --fast-list --max-transfer 700G --drive-chunk-size=1G

and my exclude is (which has a little left over but you get the idea)

felix@gemini:/opt/rclone/scripts$ cat excludes
*partial~
*sidecar
lost+found/**
torrents/**
seed/**
NZB/**

So that says move everything in all folders, but those items and folders I exclude. When it is done, it deletes any empty src folders not in the exclude so Movies/TV in my example.

1 Like

Q: why do you use /opt ? Don't you have to chown each time you add or delete a file?

I'm an old school unix kind of guy so /opt is a bit legacy but more for when people installed add applications to a server, we used that location. It's stuck with me over the years so my /opt is all my installed applications on my Linux box.

No, once you setup permissions, everything should work fine. My setup is my own non shared server so I basically run everything as a single non root user.

felix@gemini:/$ ls -al | grep opt
drwxrwxr-x  38 felix felix 4096 Mar 10 05:41 opt

felix is the user and runs everything on my server.

There are a million ways to do things and usually just do whatever works best for your brain as this works well for my use case and setup. My setup would not work well for a seed box or something shared :slight_smile:

1 Like

What do the two asterisks indicate? One means anything inside those folders but two means anything under subfolders, recursively?

The ** adds in the / so it grabs any directories down as well.

Use ** to match anything, including slashes (/).

https://rclone.org/filtering/#patterns does a pretty good job. If you have anything to help make things more helpful, we can always make updates to make it better.

I always do my filter testing with just a rclone ls command to validate it does what I want and --dry-run it and finally let it fly after those both first work.

It's usually a bit of trial and error depending on how complex the use case is as I never get it right the first time :slight_smile:

1 Like

So what filter do you think I should use to weed out mkv files under subfolders of a folder?

So I went through the wiki and moveto behaves like mv??

Can you give an example of what your structure looks like and what would be filtered? I'm more a visual person and seeing it would really help.

moveto I believe is more for moving one file and giving it a new name on the other side. I've never really used it.

Sorry, forgot to mention. --include "*.mkv" suits my needs.

But there's another issue I'm facing where I can't delete the folders after the mkv files are moved.

does '--delete-empty-src-dirs' address that?

The folders aren’t empty, so no.

This is confusing then. If the folders aren't empty how would the move delete them? You'd either have to run rclone delete and then rclone rmdirs or you'd need to just run rclone purge and the entire path will be removed. Thats how it works as of now. I'd argue that it would be nice to have a flag to leave the root directory (maybe an enhancement) or to have a flag to have rclone delete also remove the directories in one go (which is probably better actually).