What is the problem you are having with rclone?
When using ListParts for a multipart upload on rclone serve s3 with a MaxParts value strictly less than half of the total uploaded parts, rclone corrupts the pagination state.
Not only does the NextPartNumberMarker fail to advance correctly, but the PartNumber values of the returned parts are reset to 0-based indices instead of their original uploaded numbers. This leads to duplicate ETags in subsequent pages and an infinite pagination loop.
Minimal Reproducible Example (C#) and client console output showing the duplication:
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.Runtime;
const string endpoint = "http://localhost:8081";
const string accessKey = "admin";
const string secretKey = "admin";
const string bucket = "test-bug-bucket";
const string key = "test-file";
const int totalParts = 7;
const int maxParts = 3; // set < totalParts to trigger the bug
var creds = new BasicAWSCredentials(accessKey, secretKey);
var config = new AmazonS3Config
{
ServiceURL = endpoint,
ForcePathStyle = true,
UseHttp = true,
};
using var s3 = new AmazonS3Client(creds, config);
try { await s3.PutBucketAsync(bucket); } catch { }
var upload = await s3.InitiateMultipartUploadAsync(new InitiateMultipartUploadRequest
{
BucketName = bucket,
Key = key
});
string uploadId = upload.UploadId;
Console.WriteLine($"UploadId: {uploadId}");
var rnd = new Random();
for (int i = 1; i <= totalParts; i++)
{
var data = new byte[5 * 1024 * 1024];
rnd.NextBytes(data);
using var stream = new MemoryStream(data);
var resp = await s3.UploadPartAsync(new UploadPartRequest
{
BucketName = bucket,
Key = key,
UploadId = uploadId,
PartNumber = i,
InputStream = stream,
PartSize = data.Length
});
Console.WriteLine($"Uploaded part {i}, ETag={resp.ETag}");
}
Console.WriteLine($"\nListing parts with MaxParts={maxParts} (totalParts={totalParts}):");
string? marker = null;
var seenEtags = new HashSet<string>();
bool isTruncated;
int page = 1;
do
{
var listResp = await s3.ListPartsAsync(new ListPartsRequest
{
BucketName = bucket,
Key = key,
UploadId = uploadId,
MaxParts = maxParts,
PartNumberMarker = marker
});
Console.WriteLine("");
Console.WriteLine($"--- Page {page} | IsTruncated: {listResp.IsTruncated} | NextMarker: {listResp.NextPartNumberMarker}");
foreach (var part in listResp.Parts)
{
Console.Write($"Page {page}: PartNumber={part.PartNumber}, ETag={part.ETag}");
if (!seenEtags.Add(part.ETag))
Console.WriteLine(" <-- DUPLICATE ETag (pagination bug)");
else
Console.WriteLine("");
}
marker = listResp.NextPartNumberMarker?.ToString();
isTruncated = listResp.IsTruncated ?? false;
page++;
if (page > 4) break;
} while (isTruncated);
await s3.AbortMultipartUploadAsync(new AbortMultipartUploadRequest
{
BucketName = bucket,
Key = key,
UploadId = uploadId
});
Console.WriteLine("\nDone. If any ETag appears more than once, the pagination bug is confirmed.");
C:/myexample/ConsoleApp1/ConsoleApp1/bin/Debug/net10.0/ConsoleApp1.exe
UploadId: 8
Uploaded part 1, ETag="9fe358614acd2a0afbe28acfa3611ea6"
Uploaded part 2, ETag="cb4bc210ac4947ef26c939177f7c72d2"
Uploaded part 3, ETag="3c73e35516173b3b0d22e1620e22fc66"
Uploaded part 4, ETag="631a8e012cfb272d375e05093a19256f"
Uploaded part 5, ETag="2309f1c78a35f6d3d18c8bfcd13e7a81"
Uploaded part 6, ETag="b6f6dfae5cb081d9f63db7d1c95c87da"
Uploaded part 7, ETag="bba39d27210958a06709bdd84fcb42dc"
Listing parts with MaxParts=3 (totalParts=7):
--- Page 1 | IsTruncated: True | NextMarker: 4
Page 1: PartNumber=1, ETag="9fe358614acd2a0afbe28acfa3611ea6"
Page 1: PartNumber=2, ETag="cb4bc210ac4947ef26c939177f7c72d2"
Page 1: PartNumber=3, ETag="3c73e35516173b3b0d22e1620e22fc66"
--- Page 2 | IsTruncated: True | NextMarker: 3
Page 2: PartNumber=0, ETag="631a8e012cfb272d375e05093a19256f"
Page 2: PartNumber=1, ETag="2309f1c78a35f6d3d18c8bfcd13e7a81"
Page 2: PartNumber=2, ETag="b6f6dfae5cb081d9f63db7d1c95c87da"
--- Page 3 | IsTruncated: True | NextMarker: 3
Page 3: PartNumber=0, ETag="3c73e35516173b3b0d22e1620e22fc66" <-- DUPLICATE ETag (pagination bug)
Page 3: PartNumber=1, ETag="631a8e012cfb272d375e05093a19256f" <-- DUPLICATE ETag (pagination bug)
Page 3: PartNumber=2, ETag="2309f1c78a35f6d3d18c8bfcd13e7a81" <-- DUPLICATE ETag (pagination bug)
--- Page 4 | IsTruncated: True | NextMarker: 3
Page 4: PartNumber=0, ETag="3c73e35516173b3b0d22e1620e22fc66" <-- DUPLICATE ETag (pagination bug)
Page 4: PartNumber=1, ETag="631a8e012cfb272d375e05093a19256f" <-- DUPLICATE ETag (pagination bug)
Page 4: PartNumber=2, ETag="2309f1c78a35f6d3d18c8bfcd13e7a81" <-- DUPLICATE ETag (pagination bug)
Done. If any ETag appears more than once, the pagination bug is confirmed.
Process finished with exit code 0.
Run the command 'rclone version' and share the full output of the command.
by offical docker image:
rclone v1.73.5
- os/version: alpine 3.23.4 (64 bit)
- os/kernel: 6.6.87.2-microsoft-standard-WSL2 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.26.2
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Local filesystem served as S3 via Docker.
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone serve s3 :local:/data --addr :8080 -vv
The rclone config contents with secrets removed.
No config file used.
endpoint = "http://localhost:8081";
accessKey = "admin";
secretKey = "admin";
A log from the command with the -vv flag
2026/05/27 09:08:18 DEBUG : serve s3: CREATE BUCKET: test-bug-bucket
2026/05/27 09:08:18 DEBUG : serve s3: initiate multipart upload test-bug-bucket test-file
2026/05/27 09:08:18 DEBUG : serve s3: put multipart upload test-bug-bucket test-file 1
2026/05/27 09:08:19 DEBUG : serve s3: put multipart upload test-bug-bucket test-file 1
2026/05/27 09:08:19 DEBUG : serve s3: put multipart upload test-bug-bucket test-file 1
2026/05/27 09:08:19 DEBUG : serve s3: put multipart upload test-bug-bucket test-file 1
2026/05/27 09:08:19 DEBUG : serve s3: put multipart upload test-bug-bucket test-file 1
2026/05/27 09:08:19 DEBUG : serve s3: put multipart upload test-bug-bucket test-file 1
2026/05/27 09:08:19 DEBUG : serve s3: put multipart upload test-bug-bucket test-file 1
2026/05/27 09:08:19 DEBUG : serve s3: abort multipart upload test-bug-bucket test-file 1