Hi,
I would like to know how rclone is different than gsutil with very similar commands for moving files from local drive to google cloud bucket.
Can you please explain or point to a web page which provides this details?
Appreciate it.
Thanks,
gsutil looks to be a google developed tool that only works with Google Cloud Storage.
rclone is a tool that works with quite a plethora of backends and allows you to move / copy / etc but you can also mount with rclone.
A great spot for rclone is here:
1 Like
Thank you for your answer. I was trying to understand only specific to Google Cloud Storage perspective. We have very specific needs to integrate with GCP and was trying to understand if rclone can be more advantageous over gsutil. We need to transfer files from local drives to GCP bucket and need to do it multiple ways… triggered via application process, automated file watch within certain folders, move files once the transfer is done to archive folder, checksum ability, secure transfer , multi-threading support to do parallel transfer for very large files, handling of files in 10s of GBs etc and we need to build an application using gsutil, I believe, to achieve lot of such things, and trying to understand if rclone is more beneficial and has more out of the box abilities than gsutil and in what all cases. Do you have any comparison or more details related to that.
I’m not aware of any direct compare between the 2 tools as I personally never heard of the one before reading this as I don’t use Google Cloud Storage but Google Drive.
It really depends on what you are trying to do:
- rclone does moving of files and the number of the files really doesn’t matter
- rclone does checksums
- rclone is multi threaded
-rclone has a ‘backup’ folder for sync
I run my rclone scripts via cron and I’m sure other folks trigger things and work things like that. Hopefully someone more familiar with gsutil can weigh in, but I’d suggest just trying and seeing if it meets your needs.
Without getting into very specific use case, it’s hard to figure out what’s better.
1 Like
Appreciate your help on this. I will try to use both of them to figure out then. Mainly client is replacing SFTP transfers (the one we use is having lot of configurability inherently) and moving to GCP, and gsutil sounds pretty useful for the purpose. If rclone has something more on top of it, that would be ideal to use then.
rclone will let you move to s3 or sftp or azure with only changing the remote name - that is probably its main advantage.
1 Like
Thanks. So it seems per your comment, nothing specific advantage over gsutil in reference to only GCP, but as a tool rclone seems better if needs to deal with multiple cloud storages.
Or if you want to mount the storage on a Windows/Linux system and access it as a mount.
1 Like
rclone does these things
Available Commands:
about Get quota information from the remote.
authorize Remote authorization.
cachestats Print cache stats for a remote
cat Concatenates any files and sends them to stdout.
check Checks the files in the source and destination match.
cleanup Clean up the remote if possible
config Enter an interactive configuration session.
copy Copy files from source to dest, skipping already copied
copyto Copy files from source to dest, skipping already copied
copyurl Copy url content to dest.
cryptcheck Cryptcheck checks the integrity of a crypted remote.
cryptdecode Cryptdecode returns unencrypted file names.
dbhashsum Produces a Dropbox hash file for all the objects in the path.
dedupe Interactively find duplicate files and delete/rename them.
delete Remove the contents of path.
deletefile Remove a single file from remote.
genautocomplete Output completion script for a given shell.
gendocs Output markdown docs for rclone to the directory supplied.
hashsum Produces an hashsum file for all the objects in the path.
help Show help for rclone commands, flags and backends.
link Generate public link to file/folder.
listremotes List all the remotes in the config file.
ls List the objects in the path with size and path.
lsd List all directories/containers/buckets in the path.
lsf List directories and objects in remote:path formatted for parsing
lsjson List directories and objects in the path in JSON format.
lsl List the objects in path with modification time, size and path.
md5sum Produces an md5sum file for all the objects in the path.
mkdir Make the path if it doesn't already exist.
mount Mount the remote as file system on a mountpoint.
move Move files from source to dest.
moveto Move file or directory from source to dest.
ncdu Explore a remote with a text based user interface.
obscure Obscure password for use in the rclone.conf
purge Remove the path and all of its contents.
rc Run a command against a running rclone.
rcat Copies standard input to file on remote.
rcd Run rclone listening to remote control commands only.
rmdir Remove the path if empty.
rmdirs Remove empty directories under the path.
serve Serve a remote over a protocol.
settier Changes storage class/tier of objects in remote.
sha1sum Produces an sha1sum file for all the objects in the path.
size Prints the total size and number of objects in remote:path.
sync Make source and dest identical, modifying destination only.
touch Create new file or change file modification time.
tree List the contents of the remote in a tree like fashion.
version Show the version number.
Some of which like mount, tree, ncdu, serve etc are very useful but not in gsutil.
There are things gsutil does that rclone doesn’t (eg turn object versioning on and off in a bucket).
1 Like
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.