Failed to copy: Too Large Object

Help and Support
1

Hi!

I have backup process configured using rclone to swift storage. It worked fine for a long time but then started failing with the following error:

2017/03/03 09:57:20 rclone: Version “v1.35” starting with parameters ["/opt/rclone-v1.35-linux-amd64/rclone" “-v” “–log-file=/tmp/rclone.log” “–config” “/opt/rclone-v1.35-linux-amd64/.rclone.conf” “copy” “/tmp/jenkins_backup_2017-03-03_09-56.tar.gz.gpg” “selectel:backup_infra_jenkins_daily”]
2017/03/03 09:57:21 Swift container backup_infra_jenkins_daily: Modify window is 1ns
2017/03/03 09:57:21 Swift container backup_infra_jenkins_daily: Waiting for checks to finish
2017/03/03 09:57:21 Swift container backup_infra_jenkins_daily: Waiting for transfers to finish
2017/03/03 09:57:21 jenkins_backup_2017-03-03_09-56.tar.gz.gpg: Failed to copy: Too Large Object
2017/03/03 09:57:21 Attempt 1/3 failed with 1 errors and: Too Large Object
2017/03/03 09:57:21 Swift container backup_infra_jenkins_daily: Waiting for checks to finish
2017/03/03 09:57:21 Swift container backup_infra_jenkins_daily: Waiting for transfers to finish
2017/03/03 09:57:21 jenkins_backup_2017-03-03_09-56.tar.gz.gpg: Failed to copy: Too Large Object
2017/03/03 09:57:21 Attempt 2/3 failed with 1 errors and: Too Large Object
2017/03/03 09:57:21 Swift container backup_infra_jenkins_daily: Waiting for checks to finish
2017/03/03 09:57:21 Swift container backup_infra_jenkins_daily: Waiting for transfers to finish
2017/03/03 09:57:21 jenkins_backup_2017-03-03_09-56.tar.gz.gpg: Failed to copy: Too Large Object
2017/03/03 09:57:21 Attempt 3/3 failed with 1 errors and: Too Large Object
2017/03/03 09:57:21 Failed to copy: Too Large Object

It looks weird because the file size has not been changed, I’ve even decreased it to 900Mb. Before that files bigger than 2Gb were loaded successfully.
-rw-r–r-- 1 root root 930912664 Mar 3 09:57 /tmp/jenkins_backup_2017-03-03_09-56.tar.gz.gpg

Could you please advise any debug steps or workaround? I suppose that the problem can be on the storage side.

Thanks!

2

I have also contacted storage support team. They confirmed that there was a limit on files count.
I was confused with the error message, but I think this question can be closed now.

Thanks!

3

That error message is returned by the swift library on an http 413 error. Http 413 is defined as “Object too large” so it is the correct error message. It is confusing that the underlying swift would return a 413 in this case!

4

Hi @ncw,
I try move large object between 2 swift cluster backend, but the problem raised when destination's quota have free less than size of object but the quota free larger than segment's size. The first segments create normally at destination but when destination reached the quota and error raised. After that rclone continue retry but can not because the destination reached limit quota. Finally, rclone finish with error that could not move the object. I check at destination container(segment) several segments created and not delete after run.

Could you advise any solution to fix or workaround ?

Thanks

5

Please open a new post and detail your version / logging information / etc and do not bump a 2 year old post.

Thanks!