Inconsistent behavior when deleting files from a mounted S3 bucket

Help and Support
1

What is the problem you are having with rclone?

Hello everyone. I have mounted an S3 bucket as a file system using rclone. The mounting was successful. The problem comes when I upload a file from server side and then try to delete it from the S3 bucket (using the AWS interface). It is deleted from the bucket, but sometimes it is still present locally. This behavior is very inconsistent, sometimes it is deleted locally, but more often it is not. Has anyone encountered such a problem? Am I missing something?

What is your rclone version (output from rclone version)

rclone v1.53.3

  • os/arch: linux/amd64
  • go version: go1.15.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Amazon Linux release 2 (Karoo), 64 bit

Which cloud storage system are you using? (eg Google Drive)

AWS S3

The rclone config contents with secrets removed.

[rmt]
type = s3
provider = AWS
env_auth = true
region = eu-west-1
location_constraint = eu-west-1
acl = public-read
2

Can you share a debug log the issue?

3

Here is the debug log.

I have performed the following actions:

  • Mounted an s3 bucket as a local file system
  • Uploaded a file directly to the server, using an FTP client at 12:23:09
  • Deleted the file from the s3 bucket (using AWS interface) at 12:24:09

I can still see the file locally on the server, even though it's deleted on the s3 bucket.

4

Pretty sure that's a known issue that's been reported.

@darthShadow could confirm as I don't recall the exact issue offhand nor if it was fixed yet.

Your vfs-cache-age is 10 minutes by default as well. Not sure if that plays into it.

5

I've tried lowering the vfs-cache-age to 10 seconds, but the situation is the same. I've also tried waiting for a couple of hours to see if the file will eventually disappear, but it doesn't.

6

It's indeed been fixed in the latest beta so please try with that.

The related issue was this FYI:

1 Like
7

It is indeed working when using the latest beta version. Thank you.

8

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.