I 've got a list of files that I'd like to upload, but under a different name.
For a single file I'm using the command
rclone copyto -v <local file path>/test.txt <remote>:<file path>/othername.txt. For a whole list of files to be stored under a different name this tedious and slow.
I know of the --from-file flag for the source file list to be uploaded as-is, but is there something similar for uploading files under a different name as well?
There is not currently as copyto is meant for a single operation. Assuming you are generating a list of files with a from/to setup, I'd just run through a copyto loop.
That's exactly what I did, but it is extremely slow for large amounts of files. Maybe I need to copy files locally first, rename them, then upload them with "copyto --from-file" or something...
Thanks!
It depends on what you are doing and what backend/etc so it's super hard to answer as that's why we have the help and support template so we can get the info up front and try to give a complete answer.
Well basically I'm reading a list of files from a database, which also contains their names on the remote. The remote can be anything (someone else actually creates the database and defines the remote) but it's probably one of SFTP or Box.
The files are copied under their new name to directories that are also contained in the database or calculated from some other information, and might or might not exist on the remote.
As mentioned in my first post I'm currently using
rclone copyto -v <local file path>/test.txt <remote>:<file path>/othername.txt
on every file, which means that for every file a new connection and authentication has to be made in addition to the copying and renaming, which makes the whole thing last around 10 seconds per file for very small files (less than 1kb) and of course takes longer for large files (up to several Mb but usually less than 100 Mb). There can be lots of files to upload.
I'm using a pretty recent rclone (1.54.x, I think?) on Windows 10.
The remote matters in the question as some support server side move and some do not. So speed would impacted depending on what the remote is. Certain remotes are slower on file creates so lots of small files are slow on Google Drive API for example.
So if that was my use case, I'd probably tackle it via a staging area and do everything on a stage location and upload that area instead of trying to rename lots and lots of file via the rclone route as renaming that way feels much slower/cumbersome.
This is probably what I'm gonna do.
Thanks!
If you want an efficient way to run lots of rclone copyto commands then the way to do it is to start an rclone rcd and then use the API to submit operations/copyfile jobs. Using the rclone rc operations/copyfile command is convenient.
A staging area works well too 
I didn't know that rclone can be run as a service, this is definitely going to help with some of my use cases. Thank you!
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.