Split folder with many files into smaller folders

I have a folder in my Google Drive that contains many images (+200000). I want to split these files into multiple smaller directories in my Google Drive, containing 30000 images each. Is there any way to do this with rclone?

I have the drive mounted in rclone. I have rclone version v1.51.0 on Windows 10.

If you've got the directory mounted, then you can use batch commands, eg this to split them up by initial letter.

cd mountdir
mkdir a
move a* a
mkdir b
move b* b
mkdir z
move z* z

Thank you for replying! Unfortunately all the files begin with the number 1, and are followed by random numbers. All files are named like this: '1212359897939005441 2020-02-11.jpg'. Can I somehow use the * for the end of the string before .jpg? Then I could do it by date. I tried this, but it doesn't do anything:

rclone move gdrivedu:hkimages gdrivedu:hkimagesfeb --include *2019-02-11.jpg --dry-run -P -vv

Maybe I used the term 'mount' incorrectly. What I mean is that I have my Gdrive set up as a remote in rclone. Sorry, I am a bit new to this, so I just wanted to clear that up.

you might want to update to latest stable at https://rclone.org/install/

filters can be confusing. for testing, try rclone ls
rclone ls gdrivedu:hkimagesfeb --include *2019-02-11.jpg

Thanks for replying. I have updated rclone now to newest stable version.

I tried running this:
rclone ls gdrivedu:hkimages --include *2019-02-*.jpg
And succesfully got all images from february 2019 listed in the cmd prompt.

I wish to copy all those images to the 'hkimagesfeb' directory on Gdrive, so I tried:
rclone copy gdrivedu:hkimagesfeb gdrivedu:hkimages --include *2019-02-*.jpg

But it just returns:

2021/03/10 09:50:52 NOTICE:
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time:      36m0.9s

2021/03/10 09:51:04 INFO  : There was nothing to transfer

Does the filter only work with ls?

not sure exactly what you want to do.

you need to post the command with a debug log....

Sorry I am bad at communicating what I want to do. Thanks for helping. In the meantime, I have found a solution. In case anyone in the future stumbles upon this post:

I make a text file of all files in the directory that matches my pattern:
rclone lsf gdrivedu:hkimages --include *2019-02-*.jpg > hkimagesfeb.txt

Then I create a directory, and copy all files listed in the txt file to that directory:
rclone mkdir gdrivedu:hkimagesfeb
rclone copy gdrivedu:hkimages gdrivedu:hkimagesfeb --files-from hkimagesfeb.txt -P -v

rclone will create the dest folder for you.

rclone copy gdrivedu:hkimages gdrivedu:hkimagesfeb --include *2019-02-*.jpg

Oh cool, that is one step less I need to do. Thanks. I do the txt-file step because I want to be able to split it into smaller portions using Python, as some months have too many images for me to transfer them all to one folder. That way I can create sub-folders per month containing a smaller amount of images.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.