Storage Type S3 vs Swift

Help and Support
1

What is the problem you are having with rclone?

My storage vendor offers both S3 and Swift storage type. Is one of these storage types more robust than the other? (Assuming my vendor's implementations are of similar quality.)

i.e. If I can use S3 or Swift, is one "better" (in terms of accurate data sync)?

What is your rclone version (output from rclone version)

rclone v1.50.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 20.04 64-bit

Which cloud storage system are you using? (eg Google Drive)

IDrive Cloud via S3 or Swift

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy

The rclone config contents with secrets removed.

n.a. -- really

A log from the command with the -vv flag

n.a. -- really
2

whichever way you decide, really need to update rclone, that version is years old.

i would assume s3 is a better choice as many rcloners use that including me with multiple cloud providers.

3

Both call for an extra API call for metadata so for that it's a wash.

Can you test both? See which is faster?

4

There isn't much in it. I guess your service provider is using CEPH since that has an s3 and a swift module for its radowgw.

In my opinion the s3 protocol is better engineered particularly around large file (> 5GB) handling, so I'd use that given a choice. The large file handling in swift is a mess IMHO.

As far as speed, neither return metadata in directory listings as mentioned above so one won't be faster than the other I wouldn't have thought.

I've used both extensively with CEPH and they work fine!

5

Perfect. Thank you. That's precisely the sort of information I'm looking for.

FWIW, I've measured performance with both and they're similar with this provider. They're both so much faster than what I've been using that I'm having to do extensive upload-download-diff testing, in order to convince myself they're working.

1 Like
6

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.