Wasabi copy between 2 buckets error

Help and Support
1

What is the problem you are having with rclone?

Wasabi copy between 2 buckets error

Run the command 'rclone version' and share the full output of the command.

 rclone v1.69.0

- os/version: darwin 15.3.1 (64 bit)

- os/kernel: 24.3.0 (arm64)

- os/type: darwin

- os/arch: arm64 (ARMv8 compatible)

- go/version: go1.23.4

- go/linking: dynamic

- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Wasabi

rclone copy wasabi:dfatexas/Projects/2021/Q1 wasabi:dfatexas02/Projects/2021/Q1 --progress -v --transfers 4 --checkers 8 --tpslimit 15 --s3-upload-concurrency 10 --tpslimit-burst 0 --s3-disable-checksum --bwlimit 100M --metadata --disable-http-keep-alives

[DFA-wasabi]
type = s3
provider = AWS
access_key_id = XXX
secret_access_key = XXX
endpoint = s3.us-central-1.wasabisys.com
acl = private
region = us-east-1
no_check_bucket = true

[wasabi]
type = s3
provider = Wasabi
access_key_id = XXX
secret_access_key = XXX
region = us-central-1
endpoint = s3.us-central-1.wasabisys.com
acl = private

editshare@AUSAME02 ~ % rclone copy wasabi:dfatexas/Projects/2021/Q1 wasabi:dfatexas02/Projects/2021/Q1 --progress -vv --transfers 4 --checkers 8 --tpslimit 15 --s3-upload-concurrency 10 --tpslimit-burst 0 --s3-disable-checksum --bwlimit 100M --metadata --disable-http-keep-alives
2025/03/05 10:46:56 INFO  : Starting bandwidth limiter at 100Mi Byte/s
2025/03/05 10:46:56 INFO  : Starting transaction limiter: max 15 transactions/s with burst 1
2025/03/05 10:46:56 DEBUG : rclone: Version "v1.69.0" starting with parameters ["rclone" "copy" "wasabi:dfatexas/Projects/2021/Q1" "wasabi:dfatexas02/Projects/2021/Q1" "--progress" "-vv" "--transfers" "4" "--checkers" "8" "--tpslimit" "15" "--s3-upload-concurrency" "10" "--tpslimit-burst" "0" "--s3-disable-checksum" "--bwlimit" "100M" "--metadata" "--disable-http-keep-alives"]
2025/03/05 10:46:56 DEBUG : Creating backend with remote "wasabi:dfatexas/Projects/2021/Q1"
2025/03/05 10:46:56 DEBUG : Using config file from "/Users/editshare/.config/rclone/rclone.conf"
2025/03/05 10:46:56 DEBUG : wasabi: detected overridden config - adding "{DXUJQ}" suffix to name
2025/03/05 10:46:56 DEBUG : fs cache: renaming cache item "wasabi:dfatexas/Projects/2021/Q1" to be canonical "wasabi{DXUJQ}:dfatexas/Projects/2021/Q1"
2025/03/05 10:46:56 DEBUG : Creating backend with remote "wasabi:dfatexas02/Projects/2021/Q1"
2025/03/05 10:46:56 DEBUG : wasabi: detected overridden config - adding "{DXUJQ}" suffix to name
2025/03/05 10:46:56 DEBUG : fs cache: renaming cache item "wasabi:dfatexas02/Projects/2021/Q1" to be canonical "wasabi{DXUJQ}:dfatexas02/Projects/2021/Q1"
2025/03/05 10:46:57 DEBUG : ._.DS_Store: Size and modification time the same (differ by 0s, within tolerance 1ns)
2025/03/05 10:46:57 DEBUG : ._.DS_Store: Unchanged skipping
2025/03/05 10:46:59 DEBUG : Wave_Loop_9161/Charlotte_Wave_Loop_Alt_v08.aep: Size and modification time the same (differ by 0s, within tolerance 1ns)
2025/03/05 10:46:59 DEBUG : Wave_Loop_9161/Charlotte_Wave_Loop_Alt_v08.aep: Unchanged skipping
2025/03/05 10:46:59 DEBUG : Wave_Loop_9161/Charlotte_Wave_Loop_Alt_v12.aep: Size and modification time the same (differ by 0s, within tolerance 1ns)
2025/03/05 10:46:59 DEBUG : Wave_Loop_9161/Charlotte_Wave_Loop_Alt_v12.aep: Unchanged skipping
:file checks truncated
2025/03/05 11:06:59 DEBUG : S3 bucket dfatexas02 path Projects/2021/Q1: Waiting for transfers to finish
2025/03/05 11:07:07 DEBUG : pacer: low level retry 1/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: B402A96522B83613:A, HostID: 6pgOU0YISpLf/ap6NkH0uZKE4PI1eDBgEqzeMCaJi0yrbmr608fALK+ILIuhZdx9OCTtBvN8t2wE, api error InternalError: We encountered an internal error.  Please retry the operation again later.)
2025/03/05 11:07:07 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2025/03/05 11:07:19 DEBUG : pacer: low level retry 1/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 1458A5395B3AD932:A, HostID: +gJ0twnyHeiMjfYdsDqi9T4e9MzSpsbrEhcUgw9n0+VEPPMaGMSr1zSzURR3C1Z7EqUWWzolEjnS, api error InternalError: We encountered an internal error.  Please retry the operation again later.)
2025/03/05 11:07:19 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2025/03/05 11:07:29 DEBUG : pacer: low level retry 2/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 7BB5DE411DDDF0A2:A, HostID: a/IL1JlGfaBrQWiihDHn1/iVwUNbRT2aoEyjrho1MQA2Pbl70VIlcvhtptJQDT3FbsaqlHoPNewV, api error InternalError: We encountered an internal error.  Please retry the operation again later.)
2025/03/05 11:07:29 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2025/03/05 11:07:29 DEBUG : Broadcast/EVS/MP4/HD/SmallCap_ISOs-00_30-MAR-2021_HD.mp4: Received error: operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 7BB5DE411DDDF0A2:A, HostID: a/IL1JlGfaBrQWiihDHn1/iVwUNbRT2aoEyjrho1MQA2Pbl70VIlcvhtptJQDT3FbsaqlHoPNewV, api error InternalError: We encountered an internal error.  Please retry the operation again later. - low level retry 3/10
2025/03/05 11:09:38 DEBUG : pacer: low level retry 1/2 (error operation error S3: UploadPartCopy, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "https://dfatexas02.s3.us-central-1.wasabisys.com/Projects/2021/Q1/Broadcast/Pearl/012121_EMEA_Webcast/EMEA%20Webcast%20012121_Jan21_08-29-24.mp4?partNumber=1&uploadId=QtFo9qPDTHzcOVBK5kn_hOv0f56oTwZgotMh9EqJvmBPnht9KvbVQfkAe4C3Z0bPNrIziStli5cpxoo4k41qH08V_y4kU-fYEdIhYAJuZfeUClsn1qbPpODFcEi9rdiQ&x-id=UploadPartCopy": EOF)
2025/03/05 11:09:38 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2025/03/05 11:09:54 DEBUG : pacer: low level retry 1/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 3FC947E8ED38947A:A, HostID: 8n1Fb9i2+hHyyTXQzH9bQ+uCocBCV3zhZIhmH3uRjVeOAiNGTEWq4CsNHxFPkfej052xkxlOd5OM, api error InternalError: We encountered an internal error.  Please retry the operation again later.)
2025/03/05 11:09:54 DEBUG : pacer: Rate limited, increasing sleep to 160ms
2025/03/05 11:10:02 DEBUG : pacer: low level retry 2/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: B0B11F52A5044D74:A, HostID: Gl4Wl4OHIS5H7+fRnsLMRDsIRrsbSNBeT+CFbc7up30dklf/sGgulqEJugmh3YLqDwzrofwuOjiK, api error InternalError: We encountered an internal error.  Please retry the operation again later.)
2025/03/05 11:10:02 DEBUG : pacer: Rate limited, increasing sleep to 320ms
2025/03/05 11:10:02 DEBUG : Broadcast/CLT/Lawrence/Test for Wed.mov: Received error: operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: B0B11F52A5044D74:A, HostID: Gl4Wl4OHIS5H7+fRnsLMRDsIRrsbSNBeT+CFbc7up30dklf/sGgulqEJugmh3YLqDwzrofwuOjiK, api error InternalError: We encountered an internal error.  Please retry the operation again later. - low level retry 3/10
2025/03/05 11:12:31 DEBUG : pacer: low level retry 2/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 2400FE51121B44BC:B, HostID: bfXTd47ItXkubW9ArGk5xGAEWRfa3akOVHR8IBnsQXoi4dt+qlmgK5WklmgBDkZD3XSIKzn/O7vu, api error InternalError: We encountered an internal error.  Please retry the operation again later.)
2025/03/05 11:12:31 DEBUG : pacer: Rate limited, increasing sleep to 640ms
2025/03/05 11:12:31 DEBUG : Broadcast/EVS/MP4/HD/SmallCap_ISOs-00_30-MAR-2021_HD.mp4: Received error: operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 2400FE51121B44BC:B, HostID: bfXTd47ItXkubW9ArGk5xGAEWRfa3akOVHR8IBnsQXoi4dt+qlmgK5WklmgBDkZD3XSIKzn/O7vu, api error InternalError: We encountered an internal error.  Please retry the operation again later. - low level retry 4/10
2025/03/05 11:12:55 DEBUG : pacer: low level retry 1/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 2A5844E11339EFF1:B, HostID: ojmBoP+MIL5GL8CPsjS3sRXwP01Xk5SYFMdFNXawwoyhB1Q/TtP4GG+35yHGCj64B0IGKD5lNdn9, api error InternalError: We encountered an internal error.  Please retry the operation again later.)
2025/03/05 11:12:55 DEBUG : pacer: Rate limited, increasing sleep to 1.28s
2025/03/05 11:13:30 DEBUG : pacer: low level retry 2/2 (error operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 20880BEB91C3416D:B, HostID: /pdOlLX7efy4FNyC8PYc+taBxgiiSRaiCtFeTf/Jhds3XOjN3xRMxZAlxa2uLox97aKAvkfi96VM, api error InternalError: We encountered an internal error.  Please retry the operation again later.)
2025/03/05 11:13:30 DEBUG : pacer: Rate limited, increasing sleep to 2s
2025/03/05 11:13:30 DEBUG : Broadcast/EVS/MP4/HD/Chad_Edwards_Jones_15-DEC-2020_HD.mp4: Received error: operation error S3: CopyObject, exceeded maximum number of attempts, 10, https response error StatusCode: 500, RequestID: 20880BEB91C3416D:B, HostID: /pdOlLX7efy4FNyC8PYc+taBxgiiSRaiCtFeTf/Jhds3XOjN3xRMxZAlxa2uLox97aKAvkfi96VM, api error InternalError: We encountered an internal error.  Please retry the operation again later. - low level retry 1/10
2025/03/05 11:13:59 DEBUG : pacer: low level retry 2/2 (error operation error S3: UploadPartCopy, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "https://dfatexas02.s3.us-central-1.wasabisys.com/Projects/2021/Q1/Broadcast/Pearl/012121_EMEA_Webcast/EMEA%20Webcast%20012121_Jan21_08-29-24.mp4?partNumber=1&uploadId=QtFo9qPDTHzcOVBK5kn_hOv0f56oTwZgotMh9EqJvmBPnht9KvbVQfkAe4C3Z0bPNrIziStli5cpxoo4k41qH08V_y4kU-fYEdIhYAJuZfeUClsn1qbPpODFcEi9rdiQ&x-id=UploadPartCopy": EOF)
2025/03/05 11:13:59 DEBUG : Broadcast/Pearl/012121_EMEA_Webcast/EMEA Webcast 012121_Jan21_08-29-24.mp4: Cancelling multipart copy
2025/03/05 11:13:59 DEBUG : pacer: Reducing sleep to 1.5s
2025/03/05 11:13:59 DEBUG : Broadcast/Pearl/012121_EMEA_Webcast/EMEA Webcast 012121_Jan21_08-29-24.mp4: Received error: operation error S3: UploadPartCopy, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "https://dfatexas02.s3.us-central-1.wasabisys.com/Projects/2021/Q1/Broadcast/Pearl/012121_EMEA_Webcast/EMEA%20Webcast%20012121_Jan21_08-29-24.mp4?partNumber=1&uploadId=QtFo9qPDTHzcOVBK5kn_hOv0f56oTwZgotMh9EqJvmBPnht9KvbVQfkAe4C3Z0bPNrIziStli5cpxoo4k41qH08V_y4kU-fYEdIhYAJuZfeUClsn1qbPpODFcEi9rdiQ&x-id=UploadPartCopy": EOF - low level retry 2/10
2025/03/05 11:14:01 DEBUG : pacer: Reducing sleep to 1.125s
2025/03/05 11:14:01 DEBUG : Broadcast/Pearl/012121_EMEA_Webcast/EMEA Webcast 012121_Jan21_08-29-24.mp4: Starting  multipart copy with 2 parts
2025/03/05 11:14:16 DEBUG : pacer: Reducing sleep to 843.75ms
Transferred:   	          0 B / 659.728 GiB, 0%, 0 B/s, ETA -
Checks:              7748 / 7748, 100%
Transferred:            0 / 20, 0%
Elapsed time:     27m38.5s
Transferring:
 *       Broadcast/CLT/Lawrence/Test for Wed.mov:  0% /620.904Mi, 0/s, -
 * Broadcast/Pearl/012121…121_Jan21_08-29-24.mp4:  0% /5.456Gi, 0/s, -
 * Broadcast/EVS/MP4/HD/C…nes_15-DEC-2020_HD.mp4:  0% /2.523Gi, 0/s, -
 * Broadcast/EVS/MP4/HD/S…-00_30-MAR-2021_HD.mp4:  0% /3.542Gi, 0/s, -
2

welcome to the forum,

looks like an issue at wasabi. i have been using wasabi for 6+ years, on a daily basis, rarely ever have issues.

since the source and dest are the same remote and same provider, might be able to use
--server-side-across-configs
and remove
--tpslimit 15 --s3-upload-concurrency 10 --tpslimit-burst 0 --s3-disable-checksum --bwlimit 100M --disable-http-keep-alives

3

I tried with the modified command.

editshare@AUSAME02 ~ % rclone copy wasabi:dfatexas/Projects/2021/Q1 wasabi:dfatexas02/Projects/2021/Q1 --progress -v --transfers 4 --checkers 8 --bwlimit 100M --metadata --server-side-across-configs
2025/03/05 11:28:05 INFO : Starting bandwidth limiter at 100Mi Byte/s
2025/03/05 11:41:03 ERROR : Broadcast/CLT/Lawrence/Test for Wed.mov: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 2 available, 5 requested
2025/03/05 11:41:17 ERROR : Broadcast/Pearl/012121_EMEA_Webcast/EMEA Webcast 012121_Jan21_08-29-24.mp4: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 3 available, 5 requested
2025/03/05 11:41:29 ERROR : Video/Color_Bars/TranscodeTestOutput/DB PGM SQUAWK BOX BACKUP_02-NOV-2020.mxf: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 4 available, 5 requested
2025/03/05 11:41:36 ERROR : Broadcast/EVS/MP4/HD/SmallCap_ISOs-00_30-MAR-2021_HD.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 0 available, 5 requested
2025/03/05 11:41:44 ERROR : Video/Advisor _Benchmark_MKT-14671/Media/Clip/A045L005_200304UD.MXF: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 0 available, 5 requested
2025/03/05 11:41:45 ERROR : Broadcast/EVS/MP4/HD/Chad_Edwards_Jones_15-DEC-2020_HD.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 1 available, 5 requested
2025/03/05 11:41:54 ERROR : Broadcast/EVS/MP4/HD/Target Date Retirement-02_01-DEC-2020_HD.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 2 available, 5 requested
2025/03/05 11:42:02 ERROR : Broadcast/EVS/MP4/UHD/Hosting Virtual Client Events_16-DEC-2020_16-DEC-2020_UHD.mp4: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 3 available, 5 requested
2025/03/05 11:42:11 ERROR : Video/Advisor _Benchmark_MKT-14671/Media/Clip/C013L001_210202OS.MXF: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 4 available, 5 requested
2025/03/05 11:42:41 ERROR : Video/Foundations_Q1_2021/Output/Premiere/Foundations_Q1_2021_Day01.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 1 available, 5 requested
2025/03/05 11:42:46 ERROR : Broadcast/EVS/MP4/HD/citi far end recording_30-MAR-2021_HD.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 2 available, 5 requested
2025/03/05 11:42:49 ERROR : Broadcast/Pearl/020321_Deals_and_Succession_Feb03_11-27-10/Deals_and_Succession_Day2_Feb04_14-25-34.mp4: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 3 available, 5 requested
2025/03/05 11:45:00 ERROR : Video/SMA_Promo_15386/Media/C015L001_210526R6.MXF: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 3 available, 5 requested
2025/03/05 11:45:21 ERROR : Video/Sustainability_Conference_Promo/Media/BRoll/090820/Systematic_Investing-02_26-AUG-2020.mp4: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 3 available, 5 requested
2025/03/05 11:45:26 ERROR : Video/Broadcast_Motion_Design/Projects/Ae/Renders/Round2/Broadcast_OcilatingScale_120sec_Teal_blur3_loop_1.mov: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 4 available, 5 requested
2025/03/05 11:45:49 ERROR : Broadcast/Projects/040821_FI360_National_Conference_16203/Output/FI360_National_Conference_02.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 0 available, 5 requested
2025/03/05 11:46:02 ERROR : Video/_Ingest/Aaron_Alley/FS7/XDROOT/Clip/B018L053_201001PW.MXF: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 0 available, 5 requested
2025/03/05 11:46:07 ERROR : Broadcast/Projects/050521 Virtual Fixed Income/01_Fixed_Income_Factor_Investing_Philipp_Meyer_v01.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 0 available, 5 requested
2025/03/05 11:46:18 ERROR : Broadcast/Pearl/020321_Deals_and_Succession_Feb03_11-27-10/Deals_and_Succession_Feb03_11-27-10.mp4: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 0 available, 5 requested
2025/03/05 12:05:53 ERROR : Broadcast/EVS/MP4/UHD/French_APAC_Webcast-01_07-APR-2021.mxf: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 2 available, 5 requested
2025/03/05 12:05:53 ERROR : Attempt 1/3 failed with 20 errors and: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 2 available, 5 requested
2025/03/05 12:16:09 ERROR : Broadcast/CLT/Lawrence/Test for Wed.mov: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 3 available, 5 requested
2025/03/05 12:16:17 ERROR : Broadcast/EVS/MP4/HD/Chad_Edwards_Jones_15-DEC-2020_HD.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 3 available, 5 requested
2025/03/05 12:16:22 ERROR : Broadcast/EVS/MP4/HD/SmallCap_ISOs-00_30-MAR-2021_HD.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 3 available, 5 requested
2025/03/05 12:16:29 ERROR : Broadcast/Pearl/012121_EMEA_Webcast/EMEA Webcast 012121_Jan21_08-29-24.mp4: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 4 available, 5 requested
2025/03/05 12:16:38 ERROR : Broadcast/EVS/MP4/UHD/Hosting Virtual Client Events_16-DEC-2020_16-DEC-2020_UHD.mp4: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 1 available, 5 requested
2025/03/05 12:16:50 ERROR : Broadcast/EVS/MP4/HD/Target Date Retirement-02_01-DEC-2020_HD.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 3 available, 5 requested
2025/03/05 12:17:19 ERROR : Broadcast/EVS/MP4/HD/citi far end recording_30-MAR-2021_HD.mp4: Failed to copy: operation error S3: CopyObject, failed to get rate limit token, retry quota exceeded, 4 available, 5 requested
2025/03/05 12:17:25 ERROR : Video/Advisor _Benchmark_MKT-14671/Media/Clip/A045L005_200304UD.MXF: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 0 available, 5 requested
2025/03/05 12:17:39 ERROR : Broadcast/Pearl/020321_Deals_and_Succession_Feb03_11-27-10/Deals_and_Succession_Day2_Feb04_14-25-34.mp4: Failed to copy: operation error S3: UploadPartCopy, failed to get rate limit token, retry quota exceeded, 2 available, 5 requested
Transferred: 0 B / 601.046 GiB, 0%, 0 B/s, ETA -
Errors: 9 (retrying may help)
Checks: 15496 / 15496, 100%
Transferred: 0 / 11, 0%
Elapsed time: 52m6.5s
Transferring:

  • Broadcast/EVS/MP4/UHD/…ast-01_07-APR-2021.mxf: 0% /400.292Gi, 0/s, -
  • Video/Advisor _Benchma…/C013L001_210202OS.MXF: 0% /27.235Gi, 0/s, -
  • Broadcast/Pearl/020321…ion_Feb03_11-27-10.mp4: 0% /7.332Gi, 0/s, -
  • Video/Color_Bars/Trans…BACKUP_02-NOV-2020.mxf: 0% /55.782Gi, 0/s, -
4

fwiw, i just copied a file between buckets and worked fine

rclone copy wasabi01:zork/file.ext wasabi01:rnd -vv --progress
DEBUG : rclone: Version "v1.69.1" starting with parameters ["rclone" "copy" "wasabi01:zork/file.ext" "wasabi01:rnd" "-vv" "--progress"]
DEBUG : Creating backend with remote "wasabi01:zork/file.ext"
DEBUG : Using config file from "d:\\data\\rclone\\rclone.conf"
DEBUG : fs cache: renaming child cache item "wasabi01:zork/file.ext" to be canonical for parent "wasabi01:zork"
DEBUG : Creating backend with remote "wasabi01:rnd"
DEBUG : file.ext: Need to transfer - File not found at Destination
INFO  : S3 bucket rnd: Bucket "rnd" created with ACL ""
DEBUG : file.ext: md5 = 33a1df4171e649462ef9679ac207aa77 OK
INFO  : file.ext: Copied (server-side copy)
Transferred:       15.945 MiB / 15.945 MiB, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Server Side Copies:     1 @ 15.945 MiB
Elapsed time:         0.4s

make sure the user/bucket polices are good.

ok, for a deeper look at api calls,
rclone copy a single file using --dump=headers --retries=1 --low-level-retries=1
and can remove --transfers 4 --checkers 8 --bwlimit 100M --metadata

post the full output

5

both buckets have the same policy.

Here is a successful single file copy log, for some reason it puts it in a subfolder, its in the root bucket as source:

editshare@AUSAME02 ~ % rclone copy wasabi:dfatexas/121620_ABA_Wealth_v02_0.mp4 wasabi:dfatexas02/121620_ABA_Wealth_v02_0.mp4 --dump=headers --retries=1 --low-level-retries=1

2025/03/05 13:12:46 NOTICE: Automatically setting -vv as --dump is enabled

2025/03/05 13:12:46 DEBUG : rclone: Version "v1.69.0" starting with parameters ["rclone" "copy" "wasabi:dfatexas/121620_ABA_Wealth_v02_0.mp4" "wasabi:dfatexas02/121620_ABA_Wealth_v02_0.mp4" "--dump=headers" "--retries=1" "--low-level-retries=1"]

2025/03/05 13:12:46 DEBUG : Creating backend with remote "wasabi:dfatexas/121620_ABA_Wealth_v02_0.mp4"

2025/03/05 13:12:46 DEBUG : Using config file from "/Users/editshare/.config/rclone/rclone.conf"

2025/03/05 13:12:46 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.

2025/03/05 13:12:46 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:46 DEBUG : HTTP REQUEST (req 0x140000daa00)

2025/03/05 13:12:46 DEBUG : HEAD /121620_ABA_Wealth_v02_0.mp4 HTTP/1.1

Host: dfatexas.s3.us-central-1.wasabisys.com

User-Agent: rclone/v1.69.0

Accept-Encoding: identity

Amz-Sdk-Invocation-Id: a0b00a58-f3c6-4033-8123-889d2d55c369

Amz-Sdk-Request: attempt=1; max=1

Authorization: XXXX

X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

X-Amz-Date: 20250305T191246Z

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:46 DEBUG : HTTP RESPONSE (req 0x140000daa00)

2025/03/05 13:12:46 DEBUG : HTTP/1.1 200 OK

Content-Length: 589678154

Accept-Ranges: bytes

Content-Type: video/mp4

Date: Wed, 05 Mar 2025 19:12:46 GMT

Etag: "75fdabbfd9b8bf6555ce21cf7cdd4f3d-113"

Last-Modified: Wed, 05 Mar 2025 19:07:55 GMT

Server: WasabiS3/7.21.4959-2025-01-14-7f367e51f7

X-Amz-Id-2: X5EQ3qCo2P/EvZDgfcD/0FpfhgwBEmskjWtTV7wfpxYX/bqucPfTIub/fa/aQdA15c2K6YTNY9l8

X-Amz-Meta-Atime: 2021-05-25T13:58:27-05:00

X-Amz-Meta-Btime: 2021-05-25T13:58:27-05:00

X-Amz-Meta-Gid: 20

X-Amz-Meta-Md5chksum: 51nMRn++36JPk3l8a5wrtg==

X-Amz-Meta-Mode: 100600

X-Amz-Meta-Mtime: 1621969107

X-Amz-Meta-Uid: 501

X-Amz-Request-Id: 3B8362B6F715A741:B

X-Wasabi-Cm-Reference-Id: 1741201966800 38.91.42.101 ConID:4139069700/EngineConID:38698967/Core:85

2025/03/05 13:12:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:46 DEBUG : fs cache: renaming child cache item "wasabi:dfatexas/121620_ABA_Wealth_v02_0.mp4" to be canonical for parent "wasabi:dfatexas"

2025/03/05 13:12:46 DEBUG : Creating backend with remote "wasabi:dfatexas02/121620_ABA_Wealth_v02_0.mp4"

2025/03/05 13:12:46 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.

2025/03/05 13:12:46 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:46 DEBUG : HTTP REQUEST (req 0x140005b8b40)

2025/03/05 13:12:46 DEBUG : HEAD /121620_ABA_Wealth_v02_0.mp4 HTTP/1.1

Host: dfatexas02.s3.us-central-1.wasabisys.com

User-Agent: rclone/v1.69.0

Accept-Encoding: identity

Amz-Sdk-Invocation-Id: 76276e15-b9eb-4380-b96e-02b7fcf1abaa

Amz-Sdk-Request: attempt=1; max=1

Authorization: XXXX

X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

X-Amz-Date: 20250305T191246Z

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:46 DEBUG : HTTP RESPONSE (req 0x140005b8b40)

2025/03/05 13:12:46 DEBUG : HTTP/1.1 404 Not Found

Connection: close

Content-Type: application/xml

Date: Wed, 05 Mar 2025 19:12:46 GMT

Server: WasabiS3/7.21.4960-2025-01-15-3a15701bcf

X-Amz-Id-2: EuZaf0bZzBuNezNFmuickvBFF+Y3Yb4mnZjwcZI6Hk259WUIIdqJJ2psGULCjrGctpJ5/TMjKMPg

X-Amz-Request-Id: 787C993AAB3E6B38:B

X-Wasabi-Cm-Reference-Id: 1741201966860 38.91.42.101 ConID:4139069788/EngineConID:38761995/Core:29

2025/03/05 13:12:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:46 DEBUG : HTTP REQUEST (req 0x140005b9b80)

2025/03/05 13:12:46 DEBUG : HEAD /121620_ABA_Wealth_v02_0.mp4 HTTP/1.1

Host: dfatexas.s3.us-central-1.wasabisys.com

User-Agent: rclone/v1.69.0

Accept-Encoding: identity

Amz-Sdk-Invocation-Id: 21f20f79-397e-4887-b1a4-0ec487d3eea8

Amz-Sdk-Request: attempt=1; max=1

Authorization: XXXX

X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

X-Amz-Date: 20250305T191246Z

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:46 DEBUG : HTTP RESPONSE (req 0x140005b9b80)

2025/03/05 13:12:46 DEBUG : HTTP/1.1 200 OK

Content-Length: 589678154

Accept-Ranges: bytes

Content-Type: video/mp4

Date: Wed, 05 Mar 2025 19:12:46 GMT

Etag: "75fdabbfd9b8bf6555ce21cf7cdd4f3d-113"

Last-Modified: Wed, 05 Mar 2025 19:07:55 GMT

Server: WasabiS3/7.21.4959-2025-01-14-7f367e51f7

X-Amz-Id-2: N5DLzgF/li4nj2qj02PFex62gKoAJfP0p+96jCdBhCQCYNaEBgGOn34SIMqyTWIfV8GcUsHrBX+z

X-Amz-Meta-Atime: 2021-05-25T13:58:27-05:00

X-Amz-Meta-Btime: 2021-05-25T13:58:27-05:00

X-Amz-Meta-Gid: 20

X-Amz-Meta-Md5chksum: 51nMRn++36JPk3l8a5wrtg==

X-Amz-Meta-Mode: 100600

X-Amz-Meta-Mtime: 1621969107

X-Amz-Meta-Uid: 501

X-Amz-Request-Id: 7C97F918FE2ACA25:B

X-Wasabi-Cm-Reference-Id: 1741201966800 38.91.42.101 ConID:4139069700/EngineConID:38698967/Core:85

2025/03/05 13:12:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:46 DEBUG : HTTP REQUEST (req 0x140000db540)

2025/03/05 13:12:46 DEBUG : HEAD /121620_ABA_Wealth_v02_0.mp4/121620_ABA_Wealth_v02_0.mp4 HTTP/1.1

Host: dfatexas02.s3.us-central-1.wasabisys.com

User-Agent: rclone/v1.69.0

Accept-Encoding: identity

Amz-Sdk-Invocation-Id: 11744f97-d578-4778-ad62-15c0e7a27c38

Amz-Sdk-Request: attempt=1; max=1

Authorization: XXXX

X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

X-Amz-Date: 20250305T191246Z

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:46 DEBUG : HTTP RESPONSE (req 0x140000db540)

2025/03/05 13:12:46 DEBUG : HTTP/1.1 404 Not Found

Connection: close

Content-Type: application/xml

Date: Wed, 05 Mar 2025 19:12:46 GMT

Server: WasabiS3/7.21.4960-2025-01-15-3a15701bcf

X-Amz-Id-2: IRQvv0zR6V1oAOEbBV/U8sAph41lq+St2u9drpW0Zju6T6kd3KMVjHZssSv8LPz2ocWo+d6RsmcI

X-Amz-Request-Id: 2C8F05229F81D853:B

X-Wasabi-Cm-Reference-Id: 1741201966860 38.91.42.101 ConID:4139069788/EngineConID:38761995/Core:29

2025/03/05 13:12:46 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:46 DEBUG : 121620_ABA_Wealth_v02_0.mp4: Need to transfer - File not found at Destination

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:46 DEBUG : HTTP REQUEST (req 0x14000034dc0)

2025/03/05 13:12:46 DEBUG : PUT / HTTP/1.1

Host: dfatexas02.s3.us-central-1.wasabisys.com

User-Agent: rclone/v1.69.0

Content-Length: 0

Accept-Encoding: identity

Amz-Sdk-Invocation-Id: d131425e-e6f4-4435-9dd9-2f066fe8ac2b

Amz-Sdk-Request: attempt=1; max=1

Authorization: XXXX

X-Amz-Acl: private

X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

X-Amz-Date: 20250305T191246Z

2025/03/05 13:12:46 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:47 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:47 DEBUG : HTTP RESPONSE (req 0x14000034dc0)

2025/03/05 13:12:47 DEBUG : HTTP/1.1 200 OK

Content-Length: 0

Date: Wed, 05 Mar 2025 19:12:47 GMT

Location: https://dfatexas02.s3.us-central-1.wasabisys.com//

Server: WasabiS3/7.21.4960-2025-01-15-3a15701bcf

X-Amz-Id-2: BlQbWTsyhshOyhEk/MIBf/uSMwmypIm6xfbINpuY0ahbhjTaoYL0Xa5ufrMeghgzOwV22JJC7FEp

X-Amz-Request-Id: 87929ADD4B3A796B:B

X-Wasabi-Cm-Reference-Id: 1741201966860 38.91.42.101 ConID:4139069788/EngineConID:38761995/Core:29

2025/03/05 13:12:47 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:12:47 INFO : S3 bucket dfatexas02 path 121620_ABA_Wealth_v02_0.mp4: Bucket "dfatexas02" created with ACL "private"

2025/03/05 13:12:47 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:12:47 DEBUG : HTTP REQUEST (req 0x1400037ac80)

2025/03/05 13:12:47 DEBUG : PUT /121620_ABA_Wealth_v02_0.mp4/121620_ABA_Wealth_v02_0.mp4?x-id=CopyObject HTTP/1.1

Host: dfatexas02.s3.us-central-1.wasabisys.com

User-Agent: rclone/v1.69.0

Content-Length: 0

Accept-Encoding: identity

Amz-Sdk-Invocation-Id: 82a634bc-3736-4268-ad3a-c2f8591b9952

Amz-Sdk-Request: attempt=1; max=1

Authorization: XXXX

X-Amz-Acl: private

X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

X-Amz-Copy-Source: dfatexas/121620_ABA_Wealth_v02_0.mp4

X-Amz-Date: 20250305T191247Z

X-Amz-Metadata-Directive: COPY

2025/03/05 13:12:47 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:13:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:13:05 DEBUG : HTTP RESPONSE (req 0x1400037ac80)

2025/03/05 13:13:05 DEBUG : HTTP/1.1 200 OK

Content-Length: 234

Content-Type: application/xml

Date: Wed, 05 Mar 2025 19:13:05 GMT

Server: WasabiS3/7.21.4960-2025-01-15-3a15701bcf

X-Amz-Id-2: j8Ern64ooilef4O/PEvs/f48Z3XG2lFufEeHwevcK8WTaQuxBSccEAusqbc0cWoR0UJF0fdphd8x

X-Amz-Request-Id: CD154BE29628710B:B

X-Wasabi-Cm-Reference-Id: 1741201966860 38.91.42.101 ConID:4139069788/EngineConID:38761995/Core:29

2025/03/05 13:13:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:13:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:13:05 DEBUG : HTTP REQUEST (req 0x1400037b040)

2025/03/05 13:13:05 DEBUG : HEAD /121620_ABA_Wealth_v02_0.mp4/121620_ABA_Wealth_v02_0.mp4 HTTP/1.1

Host: dfatexas02.s3.us-central-1.wasabisys.com

User-Agent: rclone/v1.69.0

Accept-Encoding: identity

Amz-Sdk-Invocation-Id: 018646b0-94cd-4311-a19a-5f4021d61022

Amz-Sdk-Request: attempt=1; max=1

Authorization: XXXX

X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

X-Amz-Date: 20250305T191305Z

2025/03/05 13:13:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2025/03/05 13:13:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:13:05 DEBUG : HTTP RESPONSE (req 0x1400037b040)

2025/03/05 13:13:05 DEBUG : HTTP/1.1 200 OK

Content-Length: 589678154

Accept-Ranges: bytes

Content-Type: video/mp4

Date: Wed, 05 Mar 2025 19:13:05 GMT

Etag: "e759cc467fbedfa24f93797c6b9c2bb6"

Last-Modified: Wed, 05 Mar 2025 19:13:06 GMT

Server: WasabiS3/7.21.4960-2025-01-15-3a15701bcf

X-Amz-Id-2: XOZIYQObdvS9WziMEntghfocoNtFyhlyzJJtwk4qj4I3ksHrE7oEBmcYN9iX2cKt+7tl7xrzAsky

X-Amz-Meta-Atime: 2021-05-25T13:58:27-05:00

X-Amz-Meta-Btime: 2021-05-25T13:58:27-05:00

X-Amz-Meta-Gid: 20

X-Amz-Meta-Md5chksum: 51nMRn++36JPk3l8a5wrtg==

X-Amz-Meta-Mode: 100600

X-Amz-Meta-Mtime: 1621969107

X-Amz-Meta-Uid: 501

X-Amz-Request-Id: AD89BC3C82DDBB47:A

X-Wasabi-Cm-Reference-Id: 1741201966860 38.91.42.101 ConID:4139069788/EngineConID:38761995/Core:29

2025/03/05 13:13:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

2025/03/05 13:13:05 DEBUG : 121620_ABA_Wealth_v02_0.mp4: md5 = e759cc467fbedfa24f93797c6b9c2bb6 OK

2025/03/05 13:13:05 INFO : 121620_ABA_Wealth_v02_0.mp4: Copied (server-side copy)

2025/03/05 13:13:05 INFO :

Transferred: 562.361 MiB / 562.361 MiB, 100%, 0 B/s, ETA -

Transferred: 1 / 1, 100%

Server Side Copies: 1 @ 562.361 MiB

Elapsed time: 19.0s

2025/03/05 13:13:05 DEBUG : 9 go routines active

Thank you,

Lawrence

6

looks good.

as per rclone docs, for rclone copy, the dest must be a folder, not a filename.

rclone copy wasabi:dfatexas/121620_ABA_Wealth_v02_0.mp4 wasabi:dfatexas02

otherwise, use rclone copyto

please, block the text you post so it looks like

2025/03/05 13:13:05 INFO : 121620_ABA_Wealth_v02_0.mp4: Copied (server-side copy)

instead of

2025/03/05 13:13:05 INFO : 121620_ABA_Wealth_v02_0.mp4: Copied (server-side copy)