Access denied 403 writing to S3 bucket

Help and Support
1

What is the problem you are having with rclone?

Hi, we have a POC going on to replace the existing product and to use rclone to mount s3 buckets as windows drives and share them. I have managed to mount the drive and net share it with the other users [thanks to @asdffdsa :raised_hands: ]
but we have noticed a few issues :

To begin with the access works fine its very very slow but still works.
Writing to the shared drive works for a while and then stops working. Starts throwing 403 access denied. I can however see empty files being created in the mount and updated in S3. When i stop the mount and restart it and copy a file to the mount it shows up in S3. [ not sure if it uses some token that gets expired ]

The EC2 has the correct policy. I am using env_auth = true, acl =private
The bucket policy is fine and it has "Block all public access" but I can see that the cloud-berry-drive on the same machine can write files to the mount fine.

Secondly, empty files created in the cache do not get removed. Deleting from the cache show up again after the refresh time. So i kill the mount and relaunch, but they get picked up from S3 now.

What is your rclone version (output from rclone version)

rclone v1.55.1

  • os/type: windows
  • os/arch: amd64

winfsp-1.9.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows Server 2016, x64

Which cloud storage system are you using? (eg Google Drive)

AWS S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount "Dev_Direct:share-aws-dev/devDirect" X: --volname="devDirect" --vfs-cache-mode=full --cache-dir="G:\devDirect" --dir-cache-time=1m --log-file=C:\Temp\install\logs\rclone_poc.log --config=C:\Temp\install\shares\rclone.conf -vv

The rclone config contents with secrets removed.

[Dev_Direct]
type = s3
provider = AWS
env_auth = true
region = eu-west-2
location_constraint = eu-west-2
acl = private

A log from the command with the -vv flag

2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 NOTICE: Database Downloads/Adjustments/adjustments_20201103_0800.csv: Failed to read metadata: Forbidden: Forbidden
	status code: 403, request id: M58SJB5K1AS9TKD7, host id: W2D4rGA0fDN2hE5zVSjbqq4SaRBF/9mARI71WL6+JMJnItVCwH67Rhr0X4AyrzfnUC5MXJDfifI=
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: Releasedir: fh=0x3
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >Releasedir: errc=0
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: Releasedir: fh=0x6
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >Releasedir: errc=0
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: Releasedir: fh=0xA
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >Releasedir: errc=0
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: Opendir: 
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >OpenFile: fd=Database Downloads/Adjustments/ (r), err=<nil>
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >Opendir: errc=0, fh=0x3
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: Readdir: ofst=0, fh=0x3
2021/05/14 09:32:08 NOTICE: Database Downloads/Adjustments/adjustments_20201103_0745.csv: Failed to read metadata: Forbidden: Forbidden
	status code: 403, request id: M58SAXEKDNV3D7FR, host id: 0Jb7IMsFgUnuCoWj1IqflF9LfOwQimbug2x8xzzyuQgCg2XYEg4juV40dDxuzeQ6Wm5YpAan49I=
2021/05/14 09:32:08 NOTICE: Database Downloads/Adjustments/adjustments_20201103_0800.csv: Failed to read metadata: Forbidden: Forbidden
	status code: 403, request id: M58MNT2PZG94S8FT, host id: ei2EyG3uAGGSwFhthFruJeANRK8385xZ2KQJyFEo2fD3ItvT6TyiEtr74yuuB6ssixEiwIF8yBo=
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >Readdir: items=1204, errc=0
2021/05/14 09:32:08 DEBUG : /Database Downloads: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 DEBUG : /Database Downloads: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 DEBUG : /: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 DEBUG : /: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /: Opendir: 
2021/05/14 09:32:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/14 09:32:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/14 09:32:08 DEBUG : /: >Opendir: errc=0, fh=0x4
2021/05/14 09:32:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 DEBUG : /: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /: Releasedir: fh=0x4
2021/05/14 09:32:08 DEBUG : /: >Releasedir: errc=0
2021/05/14 09:32:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 DEBUG : /: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:08 DEBUG : /: >Getattr: errc=0
2021/05/14 09:32:08 DEBUG : /: Opendir: 
2021/05/14 09:32:08 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/14 09:32:08 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/14 09:32:08 DEBUG : /: >Opendir: errc=0, fh=0x4
2021/05/14 09:32:08 DEBUG : /: Statfs: 
2021/05/14 09:32:08 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/14 09:32:08 DEBUG : /: Releasedir: fh=0x4
2021/05/14 09:32:08 DEBUG : /: >Releasedir: errc=0
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: Releasedir: fh=0x3
2021/05/14 09:32:08 DEBUG : /Database Downloads/Adjustments: >Releasedir: errc=0
2021/05/14 09:32:15 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:15 NOTICE: Database Downloads/Adjustments/adjustments_20201103_0800.csv: Failed to read metadata: Forbidden: Forbidden
	status code: 403, request id: GHE2WKTTKQXSVC0Y, host id: xoOd0BTFueRU2jU6C3OMuGog3loXufZaFFXGVkx6+bGlzjtsBi+r02OwyZDP463yxGax1iQLjiU=
2021/05/14 09:32:15 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: >Getattr: errc=0
2021/05/14 09:32:15 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/14 09:32:15 NOTICE: Database Downloads/Adjustments/adjustments_20201103_0800.csv: Failed to read metadata: Forbidden: Forbidden
	status code: 403, request id: GHE2RACETQ5V8Y7E, host id: a0HLLvccJO0PFdOhWFH43OZ0c8OIIX9xjKYrlO+rPVw6ZjKAEcsGSu6k3yzH62Fc0NfVbfiBz+0=
2021/05/14 09:32:15 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: >Getattr: errc=0
2021/05/14 09:32:15 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: OpenEx: flags=0x0
2021/05/14 09:32:15 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/14 09:32:15 DEBUG : Database Downloads/Adjustments/adjustments_20201103_0800.csv: Open: flags=O_RDONLY
2021/05/14 09:32:15 DEBUG : Database Downloads/Adjustments/adjustments_20201103_0800.csv: newRWFileHandle: 
2021/05/14 09:32:15 DEBUG : Database Downloads/Adjustments/adjustments_20201103_0800.csv: >newRWFileHandle: err=<nil>
2021/05/14 09:32:15 DEBUG : Database Downloads/Adjustments/adjustments_20201103_0800.csv: >Open: fd=Database Downloads/Adjustments/adjustments_20201103_0800.csv (rw), err=<nil>
2021/05/14 09:32:15 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: >OpenFile: fd=Database Downloads/Adjustments/adjustments_20201103_0800.csv (rw), err=<nil>
2021/05/14 09:32:15 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: >OpenEx: errc=0, fh=0x3
2021/05/14 09:32:15 DEBUG : /Database Downloads/Adjustments/adjustments_20201103_0800.csv: Getattr: fh=0x3
2021/05/14 09:32:15 NOTICE: Database Downloads/Adjustments/adjustments_20201103_0800.csv: Failed to read metadata: Forbidden: Forbidden
2

hi,

not sure about your use-case but might try wasabi, a s3 clone, known for hot storage.
i use wasabi for recent backups and after 30 days, i move older stuff to aws s3 deep glacier.

not sure what is going on but might want to get more debug info using --dump bodies

about the cloudberry drive, for testing, i would disable it.

3

Have you enabled HEAD requests on the bucket?

Can you do a test with -vv --dump headers - that should hopefully show what is going on.

4

the logs are pretty huge as this is running against a real time system ... can i just paste parts of it ?

5

Sure, start with one of those failing transactions.

6

so the app throws this error : for the file : adjustments_20210514_1452.csv

RUN ID: 2617015

ERROR:."Copy/Move Data File(s)": System.UnauthorizedAccessException: Access to the path '\APP-DEV3\devDirect\Database Downloads\Adjustments\adjustments_20210514_1452.csv' is denied.

I am not sure if I have captured the relevant parts .. but here it goes

2021/05/14 14:50:54 DEBUG : Using config file from "C:\\Temp\\install\\shares\\rclone.conf"
2021/05/14 14:50:54 DEBUG : rclone: Version "v1.55.1" starting with parameters ["C:\\ProgramData\\chocolatey\\lib\\rclone.portable\\tools\\rclone-v1.55.1-windows-amd64\\rclone.exe" "mount" "devDirect:share-aws-dev/devDirect" "X:" "--volname=devDirect" "--vfs-cache-mode=full" "--cache-dir=G:\\devDirect" "--dir-cache-time=3m" "--log-file=C:\\Temp\\install\\logs\\rclone.log" "--config=C:\\Temp\\install\\shares\\rclone.conf" "-vv" "--dump" "bodies"]
2021/05/14 14:50:54 DEBUG : Creating backend with remote "devDirect:share-aws-dev/devDirect"
2021/05/14 14:50:54 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2021/05/14 14:50:54 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/14 14:50:54 DEBUG : HTTP REQUEST (req 0xc000412b00)
2021/05/14 14:50:54 DEBUG : HEAD /FT-SE%20Direct HTTP/1.1
Host: *****.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210514T135054Z
X-Amz-Security-Token: *******

2021/05/14 14:50:54 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/14 14:50:54 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/14 14:50:54 DEBUG : HTTP RESPONSE (req 0xc000412b00)
2021/05/14 14:50:54 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Type: application/xml
Date: Fri, 14 May 2021 13:50:54 GMT
Server: AmazonS3
X-Amz-Id-2: WsqjxPCJknntLTPTu5KOgw7hk/8Dnc5WueCjp0wlLHIyF+7DDk595s3M/eO3ejRhHLPUYauTtKc=
X-Amz-Request-Id: BQ90CZTY0ZE9JABP

2021/05/14 14:50:54 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/14 14:50:54 INFO  : S3 bucket share-aws-dev path devDirect: poll-interval is not supported by this remote
2021/05/14 14:50:54 DEBUG : vfs cache: root is "\\\\?\\G:\\devDirect\\vfs\\devDirect\\share-aws-dev\\devDirect"
2021/05/14 14:50:54 DEBUG : vfs cache: metadata root is "\\\\?\\G:\\devDirect\\vfs\\devDirect\\share-aws-dev\\devDirect"
2021/05/14 14:50:54 DEBUG : Creating backend with remote "\\\\?\\G:\\devDirect\\vfs\\devDirect\\share-aws-dev\\devDirect"
2021/05/14 14:50:54 DEBUG : fs cache: renaming cache item "\\\\?\\G:\\devDirect\\vfs\\devDirect\\share-aws-dev\\devDirect" to be canonical "//?/G:/devDirect/vfs/devDirect/share-aws-dev/devDirect"
2021/05/14 14:50:54 DEBUG : fs cache: switching user supplied name "\\\\?\\G:\\devDirect\\vfs\\devDirect\\share-aws-dev\\devDirect" for canonical name "//?/G:/devDirect/vfs/devDirect/share-aws-dev/devDirect"
2021/05/14 14:50:54 DEBUG : Network mode mounting is disabled
2021/05/14 14:50:54 DEBUG : Mounting on "X:" ("devDirect")
2021/05/14 14:50:54 DEBUG : S3 bucket share-aws-dev path devDirect: Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=devDirect"]

Relevant to the file in question its just humongous i am trying to cut it down
7

I am sure I have managed to write files to the mount which have shown up on the S3.

8

That doesn't seem related to the errors you are talking about.

Can you find the HTTP request and the corresponding response (make sure the (req 0xc000412b00) match) for the file in question adjustments_20210514_1452.csv? And anything else in the log which mentions that file name?

9

Apologies @ncw for not getting back with the correct log entries.I tried looking for the EC2 looks like it was terminated.. I will try to redo this again when I get into work and reply to this thread.

1 Like
10

I have redone the tests once again :

My rclone conf :

[PTALWAR-TEST]
type = s3
provider = AWS
region = eu-west-2
location_constraint = eu-west-2
acl = private
env_auth = true

Command to mount :

rclone mount PTALWAR-TEST:share-aws-index-dev/PTALWAR-TEST Y: --volname="PTALWAR-TEST" --vfs-cache-mode=full --no-checksum --cache-dir=G:\PTALWAR-TEST --dir-cache-time=1m --log-file=C:\Temp\install\logs\ptal_rclone.log --config=C:\Temp\install\shares\rclone.conf -vv --dump headers

Logs :

2021/05/26 18:40:03 DEBUG : Using config file from "C:\\Temp\\install\\shares\\rclone.conf"
2021/05/26 18:40:03 DEBUG : rclone: Version "v1.55.1" starting with parameters ["C:\\ProgramData\\chocolatey\\lib\\rclone.portable\\tools\\rclone-v1.55.1-windows-amd64\\rclone.exe" "mount" "PTALWAR-TEST:share-aws-index-dev/PTALWAR-TEST" "Y:" "--volname=PTALWAR-TEST" "--vfs-cache-mode=full" "--no-checksum" "--cache-dir=G:\\PTALWAR-TEST" "--dir-cache-time=1m" "--log-file=C:\\Temp\\install\\logs\\ptal_rclone.log" "--config=C:\\Temp\\install\\shares\\rclone.conf" "-vv" "--dump" "headers" "--s3-no-head=true" "--s3-no-check-bucket=true"]
2021/05/26 18:40:03 DEBUG : Creating backend with remote "PTALWAR-TEST:share-aws-index-dev/PTALWAR-TEST"
2021/05/26 18:40:03 DEBUG : PTALWAR-TEST: detected overridden config - adding "{Xcof1}" suffix to name
2021/05/26 18:40:03 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2021/05/26 18:40:03 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:03 DEBUG : HTTP REQUEST (req 0xc0002fc600)
2021/05/26 18:40:03 DEBUG : HEAD /PTALWAR-TEST HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210526T174003Z
X-Amz-Security-Token: *************************

2021/05/26 18:40:03 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:04 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:04 DEBUG : HTTP RESPONSE (req 0xc0002fc600)
2021/05/26 18:40:04 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Type: application/xml
Date: Wed, 26 May 2021 17:40:03 GMT
Server: AmazonS3
X-Amz-Id-2: AN4qJlAFCVUHsyNFH7wg9ofO6/40tketeNDTTHsAebPFUUWGB4/XEBhLR35O5qf+n9Fv/lHSiFU=
X-Amz-Request-Id: SNHXXPJ4V2CXDE9F

2021/05/26 18:40:04 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:04 DEBUG : fs cache: renaming cache item "PTALWAR-TEST:share-aws-index-dev/PTALWAR-TEST" to be canonical "PTALWAR-TEST{Xcof1}:share-aws-index-dev/PTALWAR-TEST"
2021/05/26 18:40:04 INFO  : S3 bucket share-aws-index-dev path PTALWAR-TEST: poll-interval is not supported by this remote
2021/05/26 18:40:04 DEBUG : vfs cache: root is "\\\\?\\G:\\PTALWAR-TEST\\vfs\\PTALWAR-TEST{Xcof1}\\share-aws-index-dev\\PTALWAR-TEST"
2021/05/26 18:40:04 DEBUG : vfs cache: metadata root is "\\\\?\\G:\\PTALWAR-TEST\\vfs\\PTALWAR-TEST{Xcof1}\\share-aws-index-dev\\PTALWAR-TEST"
2021/05/26 18:40:04 DEBUG : Creating backend with remote "\\\\?\\G:\\PTALWAR-TEST\\vfs\\PTALWAR-TEST{Xcof1}\\share-aws-index-dev\\PTALWAR-TEST"
2021/05/26 18:40:04 DEBUG : fs cache: renaming cache item "\\\\?\\G:\\PTALWAR-TEST\\vfs\\PTALWAR-TEST{Xcof1}\\share-aws-index-dev\\PTALWAR-TEST" to be canonical "//?/G:/PTALWAR-TEST/vfs/PTALWAR-TEST{Xcof1}/share-aws-index-dev/PTALWAR-TEST"
2021/05/26 18:40:04 DEBUG : fs cache: switching user supplied name "\\\\?\\G:\\PTALWAR-TEST\\vfs\\PTALWAR-TEST{Xcof1}\\share-aws-index-dev\\PTALWAR-TEST" for canonical name "//?/G:/PTALWAR-TEST/vfs/PTALWAR-TEST{Xcof1}/share-aws-index-dev/PTALWAR-TEST"
2021/05/26 18:40:04 DEBUG : Network mode mounting is disabled
2021/05/26 18:40:04 DEBUG : Mounting on "Y:" ("PTALWAR-TEST")
2021/05/26 18:40:04 DEBUG : S3 bucket share-aws-index-dev path PTALWAR-TEST: Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=PTALWAR-TEST"]
2021/05/26 18:40:04 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2021/05/26 18:40:04 DEBUG : S3 bucket share-aws-index-dev path PTALWAR-TEST: Init: 
2021/05/26 18:40:04 DEBUG : S3 bucket share-aws-index-dev path PTALWAR-TEST: >Init: 
2021/05/26 18:40:04 DEBUG : /: Statfs: 
2021/05/26 18:40:04 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/26 18:40:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:04 DEBUG : /: Readlink: 
2021/05/26 18:40:04 DEBUG : /: >Readlink: linkPath="", errc=-40
2021/05/26 18:40:04 DEBUG : /: Statfs: 
2021/05/26 18:40:04 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
The service rclone has been started.
2021/05/26 18:40:04 DEBUG : /System Volume Information/FVE2.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.1: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:04 DEBUG : HTTP REQUEST (req 0xc0002fc000)
2021/05/26 18:40:04 DEBUG : GET /?delimiter=%2F&encoding-type=url&max-keys=1000&prefix=PTALWAR-TEST%2F HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210526T174004Z
X-Amz-Security-Token: 
Accept-Encoding: gzip

2021/05/26 18:40:04 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:04 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:04 DEBUG : HTTP RESPONSE (req 0xc0002fc000)
2021/05/26 18:40:04 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Wed, 26 May 2021 17:40:05 GMT
Server: AmazonS3
X-Amz-Bucket-Region: eu-west-2
X-Amz-Id-2: zI9Gg22K+VnZFLcdAr7GdowT0Ny8hTPscgA5oQhHoGBm85i+ED0Ux3N0eJYCLC25N33ubFm9iR8=
X-Amz-Request-Id: SNHWDQ7TSK456FYY

2021/05/26 18:40:04 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:04 DEBUG : /System Volume Information/FVE2.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.1: >Getattr: errc=-2
2021/05/26 18:40:04 DEBUG : /System Volume Information/FVE2.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.2: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /System Volume Information/FVE2.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.2: >Getattr: errc=-2
2021/05/26 18:40:04 DEBUG : /System Volume Information/FVE.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.1: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /System Volume Information/FVE.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.1: >Getattr: errc=-2
2021/05/26 18:40:04 DEBUG : /System Volume Information/FVE.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.2: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /System Volume Information/FVE.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.2: >Getattr: errc=-2
2021/05/26 18:40:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:04 DEBUG : /: Opendir: 
2021/05/26 18:40:04 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:04 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:04 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/05/26 18:40:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:04 DEBUG : /: Releasedir: fh=0x0
2021/05/26 18:40:04 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:04 DEBUG : /: Opendir: 
2021/05/26 18:40:04 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:04 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:04 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/05/26 18:40:04 DEBUG : /: Releasedir: fh=0x0
2021/05/26 18:40:04 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:04 DEBUG : /: Opendir: 
2021/05/26 18:40:04 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:04 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:04 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/05/26 18:40:04 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:04 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:04 DEBUG : /: Releasedir: fh=0x0
2021/05/26 18:40:04 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:07 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:07 DEBUG : /autorun.inf: >Getattr: errc=-2
2021/05/26 18:40:07 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:07 DEBUG : /autorun.inf: >Getattr: errc=-2
2021/05/26 18:40:07 DEBUG : /autorun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:07 DEBUG : /autorun.inf: >Getattr: errc=-2
2021/05/26 18:40:07 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:07 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:07 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:07 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:07 DEBUG : /: Opendir: 
2021/05/26 18:40:07 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:07 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:07 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/05/26 18:40:07 DEBUG : /: Releasedir: fh=0x0
2021/05/26 18:40:07 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:07 DEBUG : /AutoRun.inf: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:07 DEBUG : /AutoRun.inf: >Getattr: errc=-2
2021/05/26 18:40:09 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:09 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:09 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:09 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:09 DEBUG : /: Opendir: 
2021/05/26 18:40:09 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:09 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:09 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/05/26 18:40:09 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:09 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:09 DEBUG : /: Statfs: 
2021/05/26 18:40:09 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/26 18:40:09 DEBUG : /: Releasedir: fh=0x0
2021/05/26 18:40:09 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:10 DEBUG : /: Opendir: 
2021/05/26 18:40:10 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:10 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:10 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/05/26 18:40:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:10 DEBUG : /: Releasedir: fh=0x0
2021/05/26 18:40:10 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:10 DEBUG : /: Opendir: 
2021/05/26 18:40:10 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:10 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:10 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/05/26 18:40:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:10 DEBUG : /: Statfs: 
2021/05/26 18:40:10 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/26 18:40:10 DEBUG : /: Releasedir: fh=0x0
2021/05/26 18:40:10 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /desktop.ini: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /desktop.ini: >Getattr: errc=-2
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x1
2021/05/26 18:40:11 DEBUG : /: Readdir: ofst=0, fh=0x0
2021/05/26 18:40:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:11 DEBUG : HTTP REQUEST (req 0xc000692700)
2021/05/26 18:40:11 DEBUG : HEAD /PTALWAR-TEST/CIS-CAT_Report.html HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210526T174011Z
X-Amz-Security-Token: 

2021/05/26 18:40:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:11 DEBUG : HTTP RESPONSE (req 0xc000692700)
2021/05/26 18:40:11 DEBUG : HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Type: text/html; charset=utf-8
Date: Wed, 26 May 2021 17:40:12 GMT
Etag: "d41d8cd98f00b204e9800998ecf8427e"
Last-Modified: Wed, 26 May 2021 16:48:33 GMT
Server: AmazonS3
X-Amz-Id-2: 4Qg5uZRd+Pu1Y1HO9+TRLBVGJXmq2RmDp76k41DIinOIYBqTLO8f68q5TWOJ4dp2UIR0GeiV5is=
X-Amz-Meta-Mtime: 1622047707.4942421
X-Amz-Request-Id: QPQM74SGYS3Y5ADS
X-Amz-Version-Id: UD74C_jtzccUd4.ielrVAGoqHaXDsZig
Content-Length: 0

2021/05/26 18:40:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:11 DEBUG : HTTP REQUEST (req 0xc0002fc200)
2021/05/26 18:40:11 DEBUG : HEAD /PTALWAR-TEST/EnvironmentUpdaterLog.txt HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210526T174011Z
X-Amz-Security-Token: 

2021/05/26 18:40:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:11 DEBUG : HTTP RESPONSE (req 0xc0002fc200)
2021/05/26 18:40:11 DEBUG : HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Wed, 26 May 2021 17:40:12 GMT
Etag: "d41d8cd98f00b204e9800998ecf8427e"
Last-Modified: Wed, 26 May 2021 17:19:49 GMT
Server: AmazonS3
X-Amz-Id-2: enAbnzl3Ssk4YKHNYf4IxI6MoJq+KzL8bvi7V+UwNIVf9O3K0RaIG0MSxGSqZgfEJZRp61B2LgU=
X-Amz-Meta-Mtime: 1622049582.9374348
X-Amz-Request-Id: QPQNYPHVHCNWFX3Q
X-Amz-Version-Id: pFPU.OQS9x0mo60pSQlkP10MDvaMBwRs
Content-Length: 0

2021/05/26 18:40:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:11 DEBUG : HTTP REQUEST (req 0xc0000cca00)
2021/05/26 18:40:11 DEBUG : HEAD /PTALWAR-TEST/Exceptions.txt HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210526T174011Z
X-Amz-Security-Token: 

2021/05/26 18:40:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/26 18:40:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:11 DEBUG : HTTP RESPONSE (req 0xc0000cca00)
2021/05/26 18:40:11 DEBUG : HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Wed, 26 May 2021 17:40:12 GMT
Etag: "d41d8cd98f00b204e9800998ecf8427e"
Last-Modified: Wed, 26 May 2021 15:43:57 GMT
Server: AmazonS3
X-Amz-Id-2: jBMRJbmAudqFd/xyoQe1mNaFG6uW4QtlXxFvaoUCEFBxXVio8C4WpUbzTSrTOFgD9EJ2YW9vJCk=
X-Amz-Meta-Mtime: 1622043831.7476963
X-Amz-Request-Id: QPQWK1G4F07VWB51
X-Amz-Version-Id: 1SOPYRiv0WcPDorI82KDmMfdT7ztaOF1
Content-Length: 0

2021/05/26 18:40:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/26 18:40:11 DEBUG : /: >Readdir: items=3, errc=0
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x1
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x0
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x1
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:11 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:11 DEBUG : /: Opendir: 
2021/05/26 18:40:11 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:11 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:11 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:11 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:11 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:16 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:16 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:16 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:16 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:16 DEBUG : /: Opendir: 
2021/05/26 18:40:16 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:16 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:16 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:16 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:16 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:16 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:16 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:16 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:16 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:16 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:16 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:16 DEBUG : /: Opendir: 
2021/05/26 18:40:16 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:16 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:16 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:16 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:16 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:16 DEBUG : /: Statfs: 
2021/05/26 18:40:16 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/26 18:40:16 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:16 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:25 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:25 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:25 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:25 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:25 DEBUG : /: Opendir: 
2021/05/26 18:40:25 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:25 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:25 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:25 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:25 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:25 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:25 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:26 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:26 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:26 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:26 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:26 DEBUG : /: Opendir: 
2021/05/26 18:40:26 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:26 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:26 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:26 DEBUG : /: Statfs: 
2021/05/26 18:40:26 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/26 18:40:26 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:26 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:40 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:40 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:40 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:40 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:40 DEBUG : /: Opendir: 
2021/05/26 18:40:40 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:40 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:40 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:40 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:40 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:40 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:40 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:40 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:40 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:40 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:40 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:40 DEBUG : /: Opendir: 
2021/05/26 18:40:40 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:40 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:40 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:40 DEBUG : /: Statfs: 
2021/05/26 18:40:40 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/26 18:40:40 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:40 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:55 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:55 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:55 DEBUG : /: Opendir: 
2021/05/26 18:40:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:55 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:55 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:55 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:55 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:40:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:55 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:55 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:40:55 DEBUG : /: >Getattr: errc=0
2021/05/26 18:40:55 DEBUG : /: Opendir: 
2021/05/26 18:40:55 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:40:55 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:40:55 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:40:55 DEBUG : /: Statfs: 
2021/05/26 18:40:55 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/26 18:40:55 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:40:55 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:41:04 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2021/05/26 18:41:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:10 DEBUG : /: Opendir: 
2021/05/26 18:41:10 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:41:10 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:41:10 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:41:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:10 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:41:10 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:41:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:10 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:10 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:10 DEBUG : /: Opendir: 
2021/05/26 18:41:10 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:41:10 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:41:10 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:41:10 DEBUG : /: Statfs: 
2021/05/26 18:41:10 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/26 18:41:10 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:41:10 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:41:25 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:25 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:25 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:25 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:25 DEBUG : /: Opendir: 
2021/05/26 18:41:25 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:41:25 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:41:25 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:41:25 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:25 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:25 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:41:25 DEBUG : /: >Releasedir: errc=0
2021/05/26 18:41:25 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:25 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:25 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2021/05/26 18:41:25 DEBUG : /: >Getattr: errc=0
2021/05/26 18:41:25 DEBUG : /: Opendir: 
2021/05/26 18:41:25 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2021/05/26 18:41:25 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2021/05/26 18:41:25 DEBUG : /: >Opendir: errc=0, fh=0x2
2021/05/26 18:41:25 DEBUG : /: Statfs: 
2021/05/26 18:41:25 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2021/05/26 18:41:25 DEBUG : /: Releasedir: fh=0x2
2021/05/26 18:41:25 DEBUG : /: >Releasedir: errc=0

I tried to copy three files, I got an error saying I do not have permissions but I can see those three 0 byte files being created in the cache and in the S3 :

image
image1493×646 13.4 KB

11

another thing is that these empty files don't disappear even when i delete them as they get refreshed now from S3 which has the 0 byte files

12

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::accn_num:root",
"arn:aws:iam::accn_num:role/dev-fulladmin-adfs-role"
]
},
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::share-aws-index-dev/",
"arn:aws:s3:::share-aws-index-dev"
]
},
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::accn_num:root",
"arn:aws:iam::accn_num:role/dev-fulladmin-adfs-role"
]
},
"Action": "s3:Put",
"Resource": [
"arn:aws:s3:::share-aws-index-dev/Dev-Direct/",
"arn:aws:s3:::share-aws-index-dev/PTALWAR-TEST/"
]
},
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::accn_num:root",
"arn:aws:iam::accn_num:role/dev-fulladmin-adfs-role"
]
},
"Action": "s3:Delete*",
"Resource": [
"arn:aws:s3:::share-aws-index-dev/Dev-Direct/",
"arn:aws:s3:::share-aws-index-dev/PTALWAR-TEST/"
]
}
]
}

13

There aren't any errors in your log. I don't see any 403 errors either.

Can you try using rclone copy to see if you can upload files with that and rclone lsf to see if you can list them and rclone copy then we can work out whether it it the mount causing a problem or not.

14

Hi, thanks for your reply. So I just tried to rclone copy a random file .. [ I have not added any optional params ]

rclone copy "C:\Temp\Install\prime_access_config.json" PTALWAR-TEST:share-aws-index-dev/PTALWAR-TEST --log-file=C:\Temp\install\logs\rclone_copy.log --config=C:\Temp\install\shares\rclone.conf -vv --dump bodies

The logs that show access denied :

2021/05/27 18:53:26 DEBUG : Using config file from "C:\\Temp\\install\\shares\\rclone.conf"
2021/05/27 18:53:26 DEBUG : rclone: Version "v1.55.1" starting with parameters ["C:\\ProgramData\\chocolatey\\lib\\rclone.portable\\tools\\rclone-v1.55.1-windows-amd64\\rclone.exe" "copy" "C:\\Temp\\Install\\prime_access_config.json" "PTALWAR-TEST:share-aws-index-dev/PTALWAR-TEST" "--log-file=C:\\Temp\\install\\logs\\rclone_copy.log" "--config=C:\\Temp\\install\\shares\\rclone.conf" "-vv" "--dump" "bodies"]
2021/05/27 18:53:26 DEBUG : Creating backend with remote "C:\\Temp\\Install\\prime_access_config.json"
2021/05/27 18:53:26 DEBUG : fs cache: adding new entry for parent of "C:\\Temp\\Install\\prime_access_config.json", "//?/C:/Temp/Install"
2021/05/27 18:53:26 DEBUG : Creating backend with remote "PTALWAR-TEST:share-aws-index-dev/PTALWAR-TEST"
2021/05/27 18:53:26 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2021/05/27 18:53:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:26 DEBUG : HTTP REQUEST (req 0xc000737e00)
2021/05/27 18:53:26 DEBUG : HEAD /PTALWAR-TEST HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210527T175326Z
X-Amz-Security-Token: 
2021/05/27 18:53:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:26 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:26 DEBUG : HTTP RESPONSE (req 0xc000737e00)
2021/05/27 18:53:26 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Type: application/xml
Date: Thu, 27 May 2021 17:53:26 GMT
Server: AmazonS3
X-Amz-Id-2: /D/GA410G2CTbGR8iUsoHgiIlD0RXeMjXFpR09uMuXTxBIBtXGlBc0Fl/lcoNPKSagSGo9Y2OgI=
X-Amz-Request-Id: CBFYTHJ3H1XM1XVF

2021/05/27 18:53:26 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:26 DEBUG : HTTP REQUEST (req 0xc0006f4200)
2021/05/27 18:53:26 DEBUG : HEAD /PTALWAR-TEST/prime_access_config.json HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210527T175326Z
X-Amz-Security-Token: 
2021/05/27 18:53:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 DEBUG : HTTP RESPONSE (req 0xc0006f4200)
2021/05/27 18:53:27 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Type: application/xml
Date: Thu, 27 May 2021 17:53:26 GMT
Server: AmazonS3
X-Amz-Id-2: DoM/ygughl2IntsClYfItzrFOCoV4VRKehqBjA269qCzLA1H5fSJ1EtqIdITfC1gDAju2olKibM=
X-Amz-Request-Id: S3D1DB6WBYSSH4EM

2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 DEBUG : prime_access_config.json: Need to transfer - File not found at Destination
2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : HTTP REQUEST (req 0xc0006f4700)
2021/05/27 18:53:27 DEBUG : PUT / HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Content-Length: 153
Authorization: XXXX
X-Amz-Acl: private
X-Amz-Content-Sha256: 45eae6f2cfced3c33c28eae76e8027a92fef08be88f823a1dfa67ef5c433666f
X-Amz-Date: 20210527T175327Z
X-Amz-Security-Token: Accept-Encoding: gzip

<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LocationConstraint>eu-west-2</LocationConstraint></CreateBucketConfiguration>
2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 DEBUG : HTTP RESPONSE (req 0xc0006f4700)
2021/05/27 18:53:27 DEBUG : HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Thu, 27 May 2021 17:53:26 GMT
Server: AmazonS3
X-Amz-Id-2: TAAm27zy/y+cE7Oyy9BIgYPlYcdR1XlthnVZUL3a3P3KvA3cY5vPpguivP7mYzoQ503pmi567Sg=
X-Amz-Request-Id: S3D1XW4SS2SV70BE

f3
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>S3D1XW4SS2SV70BE</RequestId><HostId>TAAm27zy/y+cE7Oyy9BIgYPlYcdR1XlthnVZUL3a3P3KvA3cY5vPpguivP7mYzoQ503pmi567Sg=</HostId></Error>
0

2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 ERROR : prime_access_config.json: Failed to copy: AccessDenied: Access Denied
	status code: 403, request id: S3D1XW4SS2SV70BE, host id: TAAm27zy/y+cE7Oyy9BIgYPlYcdR1XlthnVZUL3a3P3KvA3cY5vPpguivP7mYzoQ503pmi567Sg=
2021/05/27 18:53:27 ERROR : Attempt 1/3 failed with 1 errors and: AccessDenied: Access Denied
	status code: 403, request id: S3D1XW4SS2SV70BE, host id: TAAm27zy/y+cE7Oyy9BIgYPlYcdR1XlthnVZUL3a3P3KvA3cY5vPpguivP7mYzoQ503pmi567Sg=
2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : HTTP REQUEST (req 0xc0006f4e00)
2021/05/27 18:53:27 DEBUG : HEAD /PTALWAR-TEST/prime_access_config.json HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210527T175327Z
X-Amz-Security-Token: 
2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 DEBUG : HTTP RESPONSE (req 0xc0006f4e00)
2021/05/27 18:53:27 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Type: application/xml
Date: Thu, 27 May 2021 17:53:26 GMT
Server: AmazonS3
X-Amz-Id-2: 9EbqRv1NDpTpbBlIjcFdoXNmCFQ3r8bhJeF9syQX2KNapolz3W8or8DTA8ArL7X578Q1+ul6fw4=
X-Amz-Request-Id: S3D2TPHBN0BHG0DZ

2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 DEBUG : prime_access_config.json: Need to transfer - File not found at Destination
2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : HTTP REQUEST (req 0xc0006f5300)
2021/05/27 18:53:27 DEBUG : PUT / HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Content-Length: 153
Authorization: XXXX
X-Amz-Acl: private
X-Amz-Content-Sha256: 45eae6f2cfced3c33c28eae76e8027a92fef08be88f823a1dfa67ef5c433666f
X-Amz-Date: 20210527T175327Z
X-Amz-Security-Token: 
Accept-Encoding: gzip

<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LocationConstraint>eu-west-2</LocationConstraint></CreateBucketConfiguration>
2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 DEBUG : HTTP RESPONSE (req 0xc0006f5300)
2021/05/27 18:53:27 DEBUG : HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Thu, 27 May 2021 17:53:26 GMT
Server: AmazonS3
X-Amz-Id-2: SZ/qq749cvPqn5zxZotcgCv1tTiUJAuCooDq5PACTDaOsbpN5Pu8M9E/plADbec9hBy5vryYrcw=
X-Amz-Request-Id: S3DBMJ9TY54V7VXW

f3
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>S3DBMJ9TY54V7VXW</RequestId><HostId>SZ/qq749cvPqn5zxZotcgCv1tTiUJAuCooDq5PACTDaOsbpN5Pu8M9E/plADbec9hBy5vryYrcw=</HostId></Error>
0

2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 ERROR : prime_access_config.json: Failed to copy: AccessDenied: Access Denied
	status code: 403, request id: S3DBMJ9TY54V7VXW, host id: SZ/qq749cvPqn5zxZotcgCv1tTiUJAuCooDq5PACTDaOsbpN5Pu8M9E/plADbec9hBy5vryYrcw=
2021/05/27 18:53:27 ERROR : Attempt 2/3 failed with 1 errors and: AccessDenied: Access Denied
	status code: 403, request id: S3DBMJ9TY54V7VXW, host id: SZ/qq749cvPqn5zxZotcgCv1tTiUJAuCooDq5PACTDaOsbpN5Pu8M9E/plADbec9hBy5vryYrcw=
2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : HTTP REQUEST (req 0xc00082a000)
2021/05/27 18:53:27 DEBUG : HEAD /PTALWAR-TEST/prime_access_config.json HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20210527T175327Z
X-Amz-Security-Token: 2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 DEBUG : HTTP RESPONSE (req 0xc00082a000)
2021/05/27 18:53:27 DEBUG : HTTP/1.1 404 Not Found
Connection: close
Content-Type: application/xml
Date: Thu, 27 May 2021 17:53:26 GMT
Server: AmazonS3
X-Amz-Id-2: xc/h4D/LXoOMgv3HuDFPsph4Xa0q0XMpzZDdrIamUlQrD3IRd0Hz19b9nKhPMBUk5rT8kmA+8xw=
X-Amz-Request-Id: S3DAPJ7VBHW8HQCS

2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 DEBUG : prime_access_config.json: Need to transfer - File not found at Destination
2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : HTTP REQUEST (req 0xc000602700)
2021/05/27 18:53:27 DEBUG : PUT / HTTP/1.1
Host: share-aws-index-dev.s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.55.1
Content-Length: 153
Authorization: XXXX
X-Amz-Acl: private
X-Amz-Content-Sha256: 45eae6f2cfced3c33c28eae76e8027a92fef08be88f823a1dfa67ef5c433666f
X-Amz-Date: 20210527T175327Z
X-Amz-Security-Token: Accept-Encoding: gzip

<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LocationConstraint>eu-west-2</LocationConstraint></CreateBucketConfiguration>
2021/05/27 18:53:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 DEBUG : HTTP RESPONSE (req 0xc000602700)
2021/05/27 18:53:27 DEBUG : HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Thu, 27 May 2021 17:53:26 GMT
Server: AmazonS3
X-Amz-Id-2: Njs8HjSMVGu9hsFOp27h6xK67FiwrD8DZKE1Ezlqn4VDO14PIEMKfJhYaprdO/ubwLBwm1BeMq8=
X-Amz-Request-Id: S3D0N807W1NS6BNT

f3
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>S3D0N807W1NS6BNT</RequestId><HostId>Njs8HjSMVGu9hsFOp27h6xK67FiwrD8DZKE1Ezlqn4VDO14PIEMKfJhYaprdO/ubwLBwm1BeMq8=</HostId></Error>
0

2021/05/27 18:53:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 18:53:27 ERROR : prime_access_config.json: Failed to copy: AccessDenied: Access Denied
	status code: 403, request id: S3D0N807W1NS6BNT, host id: Njs8HjSMVGu9hsFOp27h6xK67FiwrD8DZKE1Ezlqn4VDO14PIEMKfJhYaprdO/ubwLBwm1BeMq8=
2021/05/27 18:53:27 ERROR : Attempt 3/3 failed with 1 errors and: AccessDenied: Access Denied
	status code: 403, request id: S3D0N807W1NS6BNT, host id: Njs8HjSMVGu9hsFOp27h6xK67FiwrD8DZKE1Ezlqn4VDO14PIEMKfJhYaprdO/ubwLBwm1BeMq8=
2021/05/27 18:53:27 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.3s

2021/05/27 18:53:27 DEBUG : 5 go routines active
2021/05/27 18:53:27 Failed to copy: AccessDenied: Access Denied
	status code: 403, request id: S3D0N807W1NS6BNT, host id: Njs8HjSMVGu9hsFOp27h6xK67FiwrD8DZKE1Ezlqn4VDO14PIEMKfJhYaprdO/ubwLBwm1BeMq8=

The bucket policy seems ok as i could add things using other methods. I am not sure about my rclone.conf setup

15

hi,
perhaps test using this policy, as it works

16

I might be doing something daft .... but it did not work with that either. I even added this bit into the IAM policy :


            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        }

Part of the log :

2021/05/27 19:32:03 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2021/05/27 19:32:03 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 19:32:03 DEBUG : HTTP RESPONSE (req 0xc0006e5400)
2021/05/27 19:32:03 DEBUG : HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Thu, 27 May 2021 18:32:02 GMT
Server: AmazonS3
X-Amz-Id-2: J4UvYyzoYOJZMM0GMA9b8v3WIQeDa8aqaezbHvKUaSn8lbP54yS9SzTPPxjS7TzZvMVOgQQrcHI=
X-Amz-Request-Id: 6PG3H97EGSC8EYVG

f3
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>6PG3H97EGSC8EYVG</RequestId><HostId>J4UvYyzoYOJZMM0GMA9b8v3WIQeDa8aqaezbHvKUaSn8lbP54yS9SzTPPxjS7TzZvMVOgQQrcHI=</HostId></Error>
0

2021/05/27 19:32:03 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2021/05/27 19:32:03 ERROR : prime_access_config.json: Failed to copy: AccessDenied: Access Denied
	status code: 403, request id: 6PG3H97EGSC8EYVG, host id: J4UvYyzoYOJZMM0GMA9b8v3WIQeDa8aqaezbHvKUaSn8lbP54yS9SzTPPxjS7TzZvMVOgQQrcHI=
2021/05/27 19:32:03 ERROR : Attempt 3/3 failed with 1 errors and: AccessDenied: Access Denied
	status code: 403, request id: 6PG3H97EGSC8EYVG, host id: J4UvYyzoYOJZMM0GMA9b8v3WIQeDa8aqaezbHvKUaSn8lbP54yS9SzTPPxjS7TzZvMVOgQQrcHI=
2021/05/27 19:32:03 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.3s

2021/05/27 19:32:03 DEBUG : 4 go routines active
2021/05/27 19:32:03 Failed to copy: AccessDenied: Access Denied
	status code: 403, request id: 6PG3H97EGSC8EYVG, host id: J4UvYyzoYOJZMM0GMA9b8v3WIQeDa8aqaezbHvKUaSn8lbP54yS9SzTPPxjS7TzZvMVOgQQrcHI=```
17

adding that bit to your very complex policy is not what i meant.

create a new user, apply the policy i linked and test.

and for what it is worth, i never use that bit.
instead, i lock the iam user to the bucket like so.

    {
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "arn:aws:s3:::bucket/*"
    }
18

did another test.
removed this and still no access denied errors

{
      "Effect": "Allow",
      "Action": "s3:ListAllMyBuckets",
      "Resource": "arn:aws:s3:::bucket/*"
    }
19

Created brand spanking new user & bucket and added that policy, same error.
Read another thread in here [from jan 29th , s3-rclone-v-1-52-0-or-after-permission-denied/21961] ...and I see @ncw commenting on using --s3-no-check-bucket
This did the trick and I could copy the file over.

Adding a file in the mount is now visible in the S3.
But deleting does not remove from S3, Is this possible?

Now I need to see if this will work with the application and keep up with the real time data feeds that will write into it.

Thanks once again, I really appreciate all your help.

20

yes, i use --s3-no-check-bucket for every command.

1 Like