How to minimals the count of minio's requst methods

I use rclone to mount a folder from an s3 context. How to remove useless calls on s3 side (ListObjectsV2). I need to open the file by path and read the data, witout checks.

rclone command:

rclone mount --config s3.conf s3context:test-data/2 /data/2 --log-level DEBUG --allow-non-empty --daemon --vfs-cache-mode full --vfs-read-ahead 16K --dir-cache-time 5m0s --poll-interval 50ms --s3-list-version 2 --s3-no-check-bucket --s3-no-head

cat /data/2/Application/Application.json

rclone config file:

[s3context]
type = s3
provider = Minio
env_auth = false
access_key_id = minio_access_key
secret_access_key = minio_secret_key
endpoint = http://minio-dev:9000
region = eu-south-1

rclone version:

root@user:# rclone version
rclone v1.59.1
- os/version: debian 11.4 (64 bit)
- os/kernel: 4.19.157 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.5
- go/linking: static
- go/tags: none

rclone console output:

2022/08/24 12:59:40 DEBUG : Creating backend with remote "s3context:test-data/2"
2022/08/24 12:59:40 DEBUG : Using config file from "s3.conf"
2022/08/24 12:59:40 DEBUG : s3context: detected overridden config - adding "{kJ_Xo}" suffix to name
2022/08/24 12:59:40 DEBUG : fs cache: renaming cache item "s3context:test-data/2" to be canonical "s3context{kJ_Xo}:test-data/2"

minio log:

2022-08-24T12:54:35.260 [200 OK] s3.ListObjectsV2 minio-dev:9000/test-data?delimiter=%2F&encoding-type=url&list-type=2&max-keys=1000&prefix=2%2F  172.17.0.4        317µs       ↑ 93 B ↓ 678 B
2022-08-24T12:54:35.262 [200 OK] s3.ListObjectsV2 minio-dev:9000/test-data?delimiter=%2F&encoding-type=url&list-type=2&max-keys=1000&prefix=2%2FApplication%2F  172.17.0.4        319µs       ↑ 93 B ↓ 1.0 KiB
2022-08-24T12:54:35.269 [200 OK] s3.HeadObject minio-dev:9000/test-data/2/Application/Application.json 172.17.0.4        371µs       ↑ 77 B ↓ 404 B
...
2022-08-24T12:56:10.447 [200 OK] s3.ListObjectsV2 minio-dev:9000/test-data?delimiter=%2F&encoding-type=url&list-type=2&max-keys=1000&prefix=2%2F  172.17.0.4        316µs       ↑ 93 B ↓ 678 B
2022-08-24T12:56:10.449 [200 OK] s3.ListObjectsV2 minio-dev:9000/test-data?delimiter=%2F&encoding-type=url&list-type=2&max-keys=1000&prefix=2%2FApplication%2F  172.17.0.4        1.28ms       ↑ 93 B ↓ 1.0 KiB
2022-08-24T12:56:10.451 [200 OK] s3.HeadObject minio-dev:9000/test-data/2/Application/Application.json 172.17.0.4        4.327ms      ↑ 77 B ↓ 404 B
2022-08-24T12:56:10.460 [206 Partial Content] s3.GetObject minio-dev:9000/test-data/2/Application/Application.json 172.17.0.4        992µs       ↑ 83 B ↓ 555 B
...
2022-08-24T12:59:44.689 [200 OK] s3.ListObjectsV2 minio-dev:9000/test-data?delimiter=%2F&encoding-type=url&list-type=2&max-keys=1000&prefix=2%2F  172.17.0.4        418µs       ↑ 93 B ↓ 678 B
2022-08-24T12:59:44.690 [200 OK] s3.ListObjectsV2 minio-dev:9000/test-data?delimiter=%2F&encoding-type=url&list-type=2&max-keys=1000&prefix=2%2FApplication%2F  172.17.0.4        389µs       ↑ 93 B ↓ 1.0 KiB
2022-08-24T12:59:44.692 [200 OK] s3.HeadObject minio-dev:9000/test-data/2/Application/Application.json 172.17.0.4        11.011ms     ↑ 77 B ↓ 404 B

Unless I'm misunderstanding something, you would have to compile your own version and remove the checks you deemed not necessary.

There are no flags to turn that off to my knowledge.

It doesn't seem like the rclone log matches the other one though as I don't see any read requests on the directory that are shown in the bottom log.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.