Help with s3-upload-concurrency

pradeep8985 · September 16, 2023, 3:08pm

STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.

What is the problem you are having with rclone?

I am trying to upload a 1GB file to minio by using s3-upload-concurrency flag. I have set it to 80 and i don't see 80 parallel chunk sessions being uploads while checking the logs.. Is this normal. or Am i missing something here?

Run the command 'rclone version' and share the full output of the command.

Which cloud storage system are you using? (eg Google Drive)

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

./rclone copy 1gfile.txt "miniotest:rclonetest/" --transfers "1" --s3-chunk-size=16M --s3-upload-concurrency=80 --s3-upload-cutoff=16M --s3-disable-http2 -vvv

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[miniotest]
type = s3
provider = Minio
access_key_id = ss
secret_access_key = ss
endpoint = http://minio.ss.svc.cluster.local
chunk_size=2G

A log from the command that you were trying to run with the `-vv` flag

2023/09/16 20:25:48 DEBUG : Creating backend with remote "1gfile.txt"
2023/09/16 20:25:48 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/09/16 20:25:48 DEBUG : fs cache: adding new entry for parent of "1gfile.txt", "/root/rclone-v1.63.1-linux-amd64"
2023/09/16 20:25:48 DEBUG : Creating backend with remote "miniotest:rclonetest/"
2023/09/16 20:25:48 DEBUG : miniotest: detected overridden config - adding "{F7TTe}" suffix to name
2023/09/16 20:25:48 DEBUG : name = "miniotest{F7TTe}", root = "rclonetest/", opt = &s3.Options{Provider:"Minio", EnvAuth:false, AccessKeyID:"ss", SecretAccessKey:"ss", Region:"", Endpoint:"http://minio.ss.svc.cluster.local", STSEndpoint:"", LocationConstraint:"", ACL:"", BucketACL:"", RequesterPays:false, ServerSideEncryption:"", SSEKMSKeyID:"", SSECustomerAlgorithm:"", SSECustomerKey:"", SSECustomerKeyBase64:"", SSECustomerKeyMD5:"", StorageClass:"", UploadCutoff:16777216, CopyCutoff:4999341932, ChunkSize:16777216, MaxUploadParts:10000, DisableChecksum:false, SharedCredentialsFile:"", Profile:"", SessionToken:"", UploadConcurrency:80, ForcePathStyle:true, V2Auth:false, UseAccelerateEndpoint:false, LeavePartsOnError:false, ListChunk:1000, ListVersion:0, ListURLEncode:fs.Tristate{Value:false, Valid:false}, NoCheckBucket:false, NoHead:false, NoHeadObject:false, Enc:0x3000002, MemoryPoolFlushTime:60000000000, MemoryPoolUseMmap:false, DisableHTTP2:true, DownloadURL:"", DirectoryMarkers:false, UseMultipartEtag:fs.Tristate{Value:false, Valid:false}, UsePresignedRequest:false, Versions:false, VersionAt:fs.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}, Decompress:false, MightGzip:fs.Tristate{Value:false, Valid:false}, UseAcceptEncodingGzip:fs.Tristate{Value:false, Valid:false}, NoSystemMetadata:false}
2023/09/16 20:25:48 DEBUG : Resolving service "s3" region "us-east-1"
2023/09/16 20:25:48 DEBUG : fs cache: renaming cache item "miniotest:rclonetest/" to be canonical "miniotest{F7TTe}:rclonetest"
2023/09/16 20:25:48 DEBUG : 1gfile.txt: Need to transfer - File not found at Destination
2023/09/16 20:25:51 DEBUG : 1gfile.txt: multipart upload starting chunk 1 size 16Mi offset 0/1Gi
2023/09/16 20:25:51 DEBUG : 1gfile.txt: multipart upload starting chunk 2 size 16Mi offset 16Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 3 size 16Mi offset 32Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 4 size 16Mi offset 48Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 5 size 16Mi offset 64Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 6 size 16Mi offset 80Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 7 size 16Mi offset 96Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 8 size 16Mi offset 112Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 9 size 16Mi offset 128Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 10 size 16Mi offset 144Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 11 size 16Mi offset 160Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 12 size 16Mi offset 176Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 13 size 16Mi offset 192Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 14 size 16Mi offset 208Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 15 size 16Mi offset 224Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 16 size 16Mi offset 240Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 17 size 16Mi offset 256Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 18 size 16Mi offset 272Mi/1Gi
2023/09/16 20:25:52 DEBUG : 1gfile.txt: multipart upload starting chunk 19 size 16Mi offset 288Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 20 size 16Mi offset 304Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 21 size 16Mi offset 320Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 22 size 16Mi offset 336Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 23 size 16Mi offset 352Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 24 size 16Mi offset 368Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 25 size 16Mi offset 384Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 26 size 16Mi offset 400Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 27 size 16Mi offset 416Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 28 size 16Mi offset 432Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 29 size 16Mi offset 448Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 30 size 16Mi offset 464Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 31 size 16Mi offset 480Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 32 size 16Mi offset 496Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 33 size 16Mi offset 512Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 34 size 16Mi offset 528Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 35 size 16Mi offset 544Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 36 size 16Mi offset 560Mi/1Gi
2023/09/16 20:25:53 DEBUG : 1gfile.txt: multipart upload starting chunk 37 size 16Mi offset 576Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 38 size 16Mi offset 592Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 39 size 16Mi offset 608Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 40 size 16Mi offset 624Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 41 size 16Mi offset 640Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 42 size 16Mi offset 656Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 43 size 16Mi offset 672Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 44 size 16Mi offset 688Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 45 size 16Mi offset 704Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 46 size 16Mi offset 720Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 47 size 16Mi offset 736Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 48 size 16Mi offset 752Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 49 size 16Mi offset 768Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 50 size 16Mi offset 784Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 51 size 16Mi offset 800Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 52 size 16Mi offset 816Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 53 size 16Mi offset 832Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 54 size 16Mi offset 848Mi/1Gi
2023/09/16 20:25:54 DEBUG : 1gfile.txt: multipart upload starting chunk 55 size 16Mi offset 864Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: multipart upload starting chunk 56 size 16Mi offset 880Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: multipart upload starting chunk 57 size 16Mi offset 896Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: multipart upload starting chunk 58 size 16Mi offset 912Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: multipart upload starting chunk 59 size 16Mi offset 928Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: multipart upload starting chunk 60 size 16Mi offset 944Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: multipart upload starting chunk 61 size 16Mi offset 960Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: multipart upload starting chunk 62 size 16Mi offset 976Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: multipart upload starting chunk 63 size 16Mi offset 992Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: multipart upload starting chunk 64 size 16Mi offset 1008Mi/1Gi
2023/09/16 20:25:55 DEBUG : 1gfile.txt: Multipart upload Etag: cd75804014f1ce3cd328f8d25869d518-64 OK
2023/09/16 20:25:55 DEBUG : 1gfile.txt: md5 = cd573cfaace07e7949bc0c46028904ff OK
2023/09/16 20:25:55 INFO  : 1gfile.txt: Copied (new)

asdffdsa · September 16, 2023, 3:15pm

the number of chunks uploaded concurrently is specified by --s3-upload-concurrency
1024MiB / 16MiB = 64 chunks

pradeep8985 · September 16, 2023, 4:28pm

Hi.. As per the doc...

The chunk sizes used in the multipart upload are specified by --s3-chunk-size and the number of chunks uploaded concurrently is specified by --s3-upload-concurrency .

So here, 64 chunks are to be uploaded simultaneously as i specified as 80 as the s3 upload concurrency, however it doesn't seem to be the case as per the logs.

kapitainsky · September 16, 2023, 4:33pm

You will see 80 chunks when you try to upload file larger than (16*80) 1280MB.

You specify in your command:

--s3-chunk-size=16M

and

--s3-upload-concurrency=80

It means that chunk size is always 16M and maximum number of chunks is 80.

You upload 1GB file so only 64 chunks is needed using your --s3-chunk-size=16M

In addition chunked upload is only used for files larger than 16MB as you also specify:

--s3-upload-cutoff=16M

BTW - be careful with scaling these numbers up as more chunks/transfers and their size use more memory. with your settings, 1 transfer x 80 x 16MB it requires 1280MB of RAM. But if you increase transfer to e.g. 10 suddenly you need 12GB or RAM to execute such transfer

pradeep8985 · September 16, 2023, 4:55pm

Thanks @kapitainsky.I do not want to see 80 chunks here, instead i wanted to see 64 chunks uploaded parallely with 64 streams even though i set it to 80

.. Apparently, I just tested with latest 1.64 version and It seems like this is fixed in this version... Here are the logs.

2023/09/16 22:19:21 DEBUG : rclone: Version "v1.64.0" starting with parameters ["/root/rclone-v1.63.1-linux-amd64/rclone" "copy" "1gtest.txt" "miniotest:rclonetest/" "--transfers" "1" "--s3-chunk-size=16M" "--s3-upload-concurrency=80" "--s3-upload-cutoff=16M" "--s3-disable-http2" "-vvv"]
2023/09/16 22:19:21 DEBUG : Creating backend with remote "1gtest.txt"
2023/09/16 22:19:21 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/09/16 22:19:21 DEBUG : fs cache: adding new entry for parent of "1gtest.txt", "/home/unixadmin/rclone-v1.63.1-linux-amd64"
2023/09/16 22:19:21 DEBUG : Creating backend with remote "miniotest:rclonetest/"
2023/09/16 22:19:21 DEBUG : miniotest: detected overridden config - adding "{F7TTe}" suffix to name
2023/09/16 22:19:21 DEBUG : Resolving service "s3" region "us-east-1"
2023/09/16 22:19:21 DEBUG : fs cache: renaming cache item "miniotest:rclonetest/" to be canonical "miniotest{F7TTe}:rclonetest"
2023/09/16 22:19:21 DEBUG : 1gtest.txt: Need to transfer - File not found at Destination
2023/09/16 22:19:24 DEBUG : 1gtest.txt: open chunk writer: started multipart upload: MjAyMWVjM2EtMWU5ZS00Y2M1LTgzMWQtYzhmMTc4YzFmMWUxLjRlZWMzYjc1LWY4ZGEtNDEyMC1iZTBiLWQ3ZGU0NjMyY2U5OQ
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: using backend concurrency of 80 instead of --multi-thread-streams 4
2023/09/16 22:19:24 DEBUG : 1gtest.txt: Starting multi-thread copy with 64 chunks of size 16Mi with 80 parallel streams
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 3/64 (33554432-50331648) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 10/64 (150994944-167772160) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 64/64 (1056964608-1073741824) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 11/64 (167772160-184549376) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 12/64 (184549376-201326592) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 13/64 (201326592-218103808) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 14/64 (218103808-234881024) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 15/64 (234881024-251658240) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 16/64 (251658240-268435456) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 17/64 (268435456-285212672) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 18/64 (285212672-301989888) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 19/64 (301989888-318767104) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 20/64 (318767104-335544320) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 21/64 (335544320-352321536) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 22/64 (352321536-369098752) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 23/64 (369098752-385875968) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 24/64 (385875968-402653184) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 25/64 (402653184-419430400) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 26/64 (419430400-436207616) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 27/64 (436207616-452984832) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 28/64 (452984832-469762048) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 29/64 (469762048-486539264) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 30/64 (486539264-503316480) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 31/64 (503316480-520093696) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 32/64 (520093696-536870912) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 33/64 (536870912-553648128) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 34/64 (553648128-570425344) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 35/64 (570425344-587202560) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 36/64 (587202560-603979776) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 37/64 (603979776-620756992) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 38/64 (620756992-637534208) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 39/64 (637534208-654311424) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 40/64 (654311424-671088640) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 41/64 (671088640-687865856) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 42/64 (687865856-704643072) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 43/64 (704643072-721420288) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 44/64 (721420288-738197504) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 5/64 (67108864-83886080) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 6/64 (83886080-100663296) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 8/64 (117440512-134217728) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 47/64 (771751936-788529152) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 4/64 (50331648-67108864) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 49/64 (805306368-822083584) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 45/64 (738197504-754974720) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 7/64 (100663296-117440512) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 46/64 (754974720-771751936) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 51/64 (838860800-855638016) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 50/64 (822083584-838860800) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 9/64 (134217728-150994944) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 52/64 (855638016-872415232) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 48/64 (788529152-805306368) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 1/64 (0-16777216) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 58/64 (956301312-973078528) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 57/64 (939524096-956301312) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 59/64 (973078528-989855744) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 53/64 (872415232-889192448) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 54/64 (889192448-905969664) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 55/64 (905969664-922746880) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 60/64 (989855744-1006632960) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 2/64 (16777216-33554432) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 61/64 (1006632960-1023410176) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 63/64 (1040187392-1056964608) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 56/64 (922746880-939524096) size 16Mi starting
2023/09/16 22:19:24 DEBUG : 1gtest.txt: multi-thread copy: chunk 62/64 (1023410176-1040187392) size 16Mi starting

system · October 16, 2023, 4:55pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.