Rclone and internet archive, just doesnt works

STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.

Which cloud storage system are you using? (eg Google Drive)

internet archive (s3)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy /tmp/1/ archive: --metadata --header "X-Archive-Meta-Mediatype: audio" --header "X-Archive-Meta-title: Hey This Works" --header "X-Archive-Meta-Creator: It's Me"  -vv --dump headers

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

 rclone config redacted
[archive]
type = internetarchive
access_key_id = XXX
secret_access_key = XXX

A log from the command that you were trying to run with the -vv flag

rclone copy /tmp/1/ archive: --metadata --header "X-Archive-Meta-Mediatype: audio" --header "X-Archive-Meta-title: Hey This Works" --header "X-Archive-Meta-Creator: It's Me"  -vv --dump headers
2024/05/05 23:33:40 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "copy" "/tmp/1/" "archive:" "--metadata" "--header" "X-Archive-Meta-Mediatype: audio" "--header" "X-Archive-Meta-title: Hey This Works" "--header" "X-Archive-Meta-Creator: It's Me" "-vv" "--dump" "headers"]
2024/05/05 23:33:40 DEBUG : Creating backend with remote "/tmp/1/"
2024/05/05 23:33:40 DEBUG : Using config file from "/home/toshiba/.config/rclone/rclone.conf"
2024/05/05 23:33:40 DEBUG : fs cache: renaming cache item "/tmp/1/" to be canonical "/tmp/1"
2024/05/05 23:33:40 DEBUG : Creating backend with remote "archive:"
2024/05/05 23:33:40 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/05/05 23:33:40 DEBUG : test: Making directory
2024/05/05 23:33:40 DEBUG : testtt: Need to transfer - File not found at Destination
2024/05/05 23:33:40 DEBUG : Internet Archive root: Waiting for checks to finish
2024/05/05 23:33:40 DEBUG : Internet Archive root: Waiting for transfers to finish
2024/05/05 23:33:40 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:40 DEBUG : HTTP REQUEST (req 0xc00079a360)
2024/05/05 23:33:40 DEBUG : PUT /testtt HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 8
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T23:19:06.607754927+02:00
X-Amz-Filemeta-Btime: 2024-05-05T14:14:22.373242511+02:00
X-Amz-Filemeta-Gid: 1000
X-Amz-Filemeta-Mode: 100644
X-Amz-Filemeta-Rclone-Mtime: 2024-05-05T23:19:01.987690118+02:00
X-Amz-Filemeta-Rclone-Update-Track: yujidas7lixaviy6gixexuw2wararam1
X-Amz-Filemeta-Uid: 1000
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 8
Accept-Encoding: gzip

2024/05/05 23:33:40 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>


2024/05/05 23:33:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:51 DEBUG : HTTP RESPONSE (req 0xc00079a360)
2024/05/05 23:33:51 DEBUG : HTTP/1.1 500 Internal Server Error
Connection: close
Content-Length: 731
Accept-Ranges: bytes
Access-Control-Allow-Headers: authorization,x-amz-acl,x-amz-auto-make-bucket,cache-control,x-requested-with,x-file-name,x-file-size,x-archive-ignore-preexisting-bucket,x-archive-interactive-priority,x-archive-meta-title,x-archive-meta-description,x-archive-meta-language,x-archive-meta-mediatype,x-archive-meta01-subject,x-archive-meta02-subject,x-archive-meta03-subject,x-archive-meta04-subject,x-archive-meta05-subject,x-archive-meta01-collection,x-archive-meta02-collection
Access-Control-Allow-Methods: GET,POST,PUT,DELETE
Access-Control-Allow-Origin: *
Date: Sun, 05 May 2024 21:33:40 GMT
Server: Apache/2.4.41 (Ubuntu)

2024/05/05 23:33:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:51 DEBUG : pacer: low level retry 1/10 (error HTTP error 500 (500 Internal Server Error) returned body: "<?xml version='1.0' encoding='UTF-8'?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><Resource>uploadItem.py: uploading id = testtt using contribSubmit!\nlocal dir = /1/incoming/REMOTE_SUBMIT/s3-98c9150f-94cb-428a-9734-35232872c6c0-testtt\nsubmitter = oldsailfishimages@protonmail.com\npriority = -5\nscribe = None\nstubMeta = False\ndeleteAfter = True\nderive = False\nupdate = False\ncomment = s3-put\ns3_keep_old_version = False\nitem_size_kb = None\ncomplete_multipart_upload_id = None\ncomplete_multipart_upload_filename = None\nfilesize = None\nChecking for identifier availability...\ncode = not_available</Resource><RequestId>46b29e8f-897f-4570-b14b-5d406aed4ca4</RequestId></Error>")
2024/05/05 23:33:51 DEBUG : pacer: Rate limited, increasing sleep to 10ms
2024/05/05 23:33:51 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:51 DEBUG : HTTP REQUEST (req 0xc00077efc0)
2024/05/05 23:33:51 DEBUG : PUT /testtt HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 8
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T23:19:06.607754927+02:00
X-Amz-Filemeta-Btime: 2024-05-05T14:14:22.373242511+02:00
X-Amz-Filemeta-Gid: 1000
X-Amz-Filemeta-Mode: 100644
X-Amz-Filemeta-Rclone-Mtime: 2024-05-05T23:19:01.987690118+02:00
X-Amz-Filemeta-Rclone-Update-Track: yujidas7lixaviy6gixexuw2wararam1
X-Amz-Filemeta-Uid: 1000
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 8
Accept-Encoding: gzip

2024/05/05 23:33:51 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:51 DEBUG : HTTP RESPONSE (req 0xc00077efc0)
2024/05/05 23:33:51 DEBUG : Error: net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0
2024/05/05 23:33:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:51 DEBUG : pacer: low level retry 2/10 (error Put "https://s3.us.archive.org/testtt": net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0)
2024/05/05 23:33:51 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2024/05/05 23:33:51 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:51 DEBUG : HTTP REQUEST (req 0xc0002a8000)
2024/05/05 23:33:51 DEBUG : PUT /testtt HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 8
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T23:19:06.607754927+02:00
X-Amz-Filemeta-Btime: 2024-05-05T14:14:22.373242511+02:00
X-Amz-Filemeta-Gid: 1000
X-Amz-Filemeta-Mode: 100644
X-Amz-Filemeta-Rclone-Mtime: 2024-05-05T23:19:01.987690118+02:00
X-Amz-Filemeta-Rclone-Update-Track: yujidas7lixaviy6gixexuw2wararam1
X-Amz-Filemeta-Uid: 1000
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 8
Accept-Encoding: gzip

2024/05/05 23:33:51 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:51 DEBUG : HTTP RESPONSE (req 0xc0002a8000)
2024/05/05 23:33:51 DEBUG : Error: net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0
2024/05/05 23:33:51 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:51 DEBUG : pacer: low level retry 3/10 (error Put "https://s3.us.archive.org/testtt": net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0)
2024/05/05 23:33:51 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2024/05/05 23:33:51 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:51 DEBUG : HTTP REQUEST (req 0xc00050e900)
2024/05/05 23:33:51 DEBUG : PUT /testtt HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 8
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T23:19:06.607754927+02:00
X-Amz-Filemeta-Btime: 2024-05-05T14:14:22.373242511+02:00
X-Amz-Filemeta-Gid: 1000
X-Amz-Filemeta-Mode: 100644
X-Amz-Filemeta-Rclone-Mtime: 2024-05-05T23:19:01.987690118+02:00
X-Amz-Filemeta-Rclone-Update-Track: yujidas7lixaviy6gixexuw2wararam1
X-Amz-Filemeta-Uid: 1000
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 8
Accept-Encoding: gzip

2024/05/05 23:33:51 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:52 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:52 DEBUG : HTTP RESPONSE (req 0xc00050e900)
2024/05/05 23:33:52 DEBUG : Error: net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0
2024/05/05 23:33:52 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:52 DEBUG : pacer: low level retry 4/10 (error Put "https://s3.us.archive.org/testtt": net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0)
2024/05/05 23:33:52 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2024/05/05 23:33:52 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:52 DEBUG : HTTP REQUEST (req 0xc00065a360)
2024/05/05 23:33:52 DEBUG : PUT /testtt HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 8
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T23:19:06.607754927+02:00
X-Amz-Filemeta-Btime: 2024-05-05T14:14:22.373242511+02:00
X-Amz-Filemeta-Gid: 1000
X-Amz-Filemeta-Mode: 100644
X-Amz-Filemeta-Rclone-Mtime: 2024-05-05T23:19:01.987690118+02:00
X-Amz-Filemeta-Rclone-Update-Track: yujidas7lixaviy6gixexuw2wararam1
X-Amz-Filemeta-Uid: 1000
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 8
Accept-Encoding: gzip

2024/05/05 23:33:52 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:52 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:52 DEBUG : HTTP RESPONSE (req 0xc00065a360)
2024/05/05 23:33:52 DEBUG : Error: net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0
2024/05/05 23:33:52 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:52 DEBUG : pacer: low level retry 5/10 (error Put "https://s3.us.archive.org/testtt": net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0)
2024/05/05 23:33:52 DEBUG : pacer: Rate limited, increasing sleep to 160ms
2024/05/05 23:33:52 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:52 DEBUG : HTTP REQUEST (req 0xc00065aa20)
2024/05/05 23:33:52 DEBUG : PUT /testtt HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 8
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T23:19:06.607754927+02:00
X-Amz-Filemeta-Btime: 2024-05-05T14:14:22.373242511+02:00
X-Amz-Filemeta-Gid: 1000
X-Amz-Filemeta-Mode: 100644
X-Amz-Filemeta-Rclone-Mtime: 2024-05-05T23:19:01.987690118+02:00
X-Amz-Filemeta-Rclone-Update-Track: yujidas7lixaviy6gixexuw2wararam1
X-Amz-Filemeta-Uid: 1000
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 8
Accept-Encoding: gzip

2024/05/05 23:33:52 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:52 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:52 DEBUG : HTTP RESPONSE (req 0xc00065aa20)
2024/05/05 23:33:52 DEBUG : Error: net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0
2024/05/05 23:33:52 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:52 DEBUG : pacer: low level retry 6/10 (error Put "https://s3.us.archive.org/testtt": net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0)
2024/05/05 23:33:52 DEBUG : pacer: Rate limited, increasing sleep to 320ms
2024/05/05 23:33:52 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:52 DEBUG : HTTP REQUEST (req 0xc00065aea0)
2024/05/05 23:33:52 DEBUG : PUT /testtt HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 8
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T23:19:06.607754927+02:00
X-Amz-Filemeta-Btime: 2024-05-05T14:14:22.373242511+02:00
X-Amz-Filemeta-Gid: 1000
X-Amz-Filemeta-Mode: 100644
X-Amz-Filemeta-Rclone-Mtime: 2024-05-05T23:19:01.987690118+02:00
X-Amz-Filemeta-Rclone-Update-Track: yujidas7lixaviy6gixexuw2wararam1
X-Amz-Filemeta-Uid: 1000
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 8
Accept-Encoding: gzip

2024/05/05 23:33:52 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:53 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:53 DEBUG : HTTP RESPONSE (req 0xc00065aea0)
2024/05/05 23:33:53 DEBUG : Error: net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0
2024/05/05 23:33:53 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:53 DEBUG : pacer: low level retry 7/10 (error Put "https://s3.us.archive.org/testtt": net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0)
2024/05/05 23:33:53 DEBUG : pacer: Rate limited, increasing sleep to 640ms
2024/05/05 23:33:53 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:53 DEBUG : HTTP REQUEST (req 0xc00050fd40)
2024/05/05 23:33:53 DEBUG : PUT /testtt HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 8
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T23:19:06.607754927+02:00
X-Amz-Filemeta-Btime: 2024-05-05T14:14:22.373242511+02:00
X-Amz-Filemeta-Gid: 1000
X-Amz-Filemeta-Mode: 100644
X-Amz-Filemeta-Rclone-Mtime: 2024-05-05T23:19:01.987690118+02:00
X-Amz-Filemeta-Rclone-Update-Track: yujidas7lixaviy6gixexuw2wararam1
X-Amz-Filemeta-Uid: 1000
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 8
Accept-Encoding: gzip

2024/05/05 23:33:53 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:53 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:53 DEBUG : HTTP RESPONSE (req 0xc00050fd40)
2024/05/05 23:33:53 DEBUG : Error: net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0
2024/05/05 23:33:53 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:53 DEBUG : pacer: low level retry 8/10 (error Put "https://s3.us.archive.org/testtt": net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0)
2024/05/05 23:33:53 DEBUG : pacer: Rate limited, increasing sleep to 1.28s
2024/05/05 23:33:53 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:53 DEBUG : HTTP REQUEST (req 0xc0008a5d40)
2024/05/05 23:33:53 DEBUG : PUT /testtt HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 8
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T23:19:06.607754927+02:00
X-Amz-Filemeta-Btime: 2024-05-05T14:14:22.373242511+02:00
X-Amz-Filemeta-Gid: 1000
X-Amz-Filemeta-Mode: 100644
X-Amz-Filemeta-Rclone-Mtime: 2024-05-05T23:19:01.987690118+02:00
X-Amz-Filemeta-Rclone-Update-Track: yujidas7lixaviy6gixexuw2wararam1
X-Amz-Filemeta-Uid: 1000
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 8
Accept-Encoding: gzip

2024/05/05 23:33:53 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 23:33:54 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:54 DEBUG : HTTP RESPONSE (req 0xc0008a5d40)
2024/05/05 23:33:54 DEBUG : Error: net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0
2024/05/05 23:33:54 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 23:33:54 DEBUG : pacer: low level retry 9/10 (error Put "https://s3.us.archive.org/testtt": net/http: HTTP/1.x transport connection broken: http: ContentLength=8 with Body length 0)
2024/05/05 23:33:54 DEBUG : pacer: Rate limited, increasing sleep to 2s


have searched a lot this afternoon how aboutto solve this issue, updating rclone, config file etc :

forum.rclone. /t/archive-org-internetarchive-issue-setting-metadata-when-uploading/45210
forum.rclone. /t/how-to-add-metadata-in-internet-archive-archive-org/35385

An error occurred: Sorry, new users can only put 2 links in a post.

thank you for this too, discourse :slight_smile: :slight_smile: : )

if somebody have the fix to upload directories with rclone to archive.org.. thank you!

rclone copy d:\files\zork iarchive:zorkzorkzork -vv --dump headers --metadata --header "X-Archive-Meta-Mediatype: audio" --header "X-Archive-Meta-title: Hey This Works" --header "X-Archive-Meta-Creator: It's Me" 
2024/05/05 17:51:08 DEBUG : rclone: Version "v1.66.0" starting with parameters ["c:\\data\\rclone\\rclone.exe" "copy" "d:\\files\\zork" "iarchive:zorkzorkzork" "-vv" "--dump" "headers" "--metadata" "--header" "X-Archive-Meta-Mediatype: audio" "--header" "X-Archive-Meta-title: Hey This Works" "--header" "X-Archive-Meta-Creator: It's Me"]
2024/05/05 17:51:08 DEBUG : Creating backend with remote "d:\\files\\zork"
2024/05/05 17:51:08 DEBUG : Using config file from "c:\\data\\rclone\\rclone.conf"
2024/05/05 17:51:08 DEBUG : fs cache: renaming cache item "d:\\files\\zork" to be canonical "//?/d:/files/zork"
2024/05/05 17:51:08 DEBUG : Creating backend with remote "iarchive:zorkzorkzork"
2024/05/05 17:51:08 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/05/05 17:51:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 17:51:08 DEBUG : HTTP REQUEST (req 0xc00099eb40)
2024/05/05 17:51:08 DEBUG : GET /metadata/zorkzorkzork HTTP/1.1
Host: archive.org
User-Agent: rclone/v1.66.0
Authorization: XXXX
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
Accept-Encoding: gzip

2024/05/05 17:51:08 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 17:51:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 17:51:11 DEBUG : HTTP RESPONSE (req 0xc00099eb40)
2024/05/05 17:51:11 DEBUG : HTTP/2.0 200 OK
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Sun, 05 May 2024 21:51:12 GMT
Onion-Location: https://archive6zg5vrdwm4ljllgxleekeoj43lqayscd4d4kmhnyblq4h3ead.onion/metadata/zorkzorkzork
Referrer-Policy: no-referrer-when-downgrade
Server: nginx/1.25.1
Strict-Transport-Security: max-age=15724800
Vary: Accept-Encoding

2024/05/05 17:51:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 17:51:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 17:51:11 DEBUG : HTTP REQUEST (req 0xc00099f200)
2024/05/05 17:51:11 DEBUG : GET /metadata/zorkzorkzork HTTP/1.1
Host: archive.org
User-Agent: rclone/v1.66.0
Authorization: XXXX
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
Accept-Encoding: gzip

2024/05/05 17:51:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 17:51:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 17:51:11 DEBUG : HTTP RESPONSE (req 0xc00099f200)
2024/05/05 17:51:11 DEBUG : HTTP/2.0 200 OK
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Sun, 05 May 2024 21:51:12 GMT
Onion-Location: https://archive6zg5vrdwm4ljllgxleekeoj43lqayscd4d4kmhnyblq4h3ead.onion/metadata/zorkzorkzork
Referrer-Policy: no-referrer-when-downgrade
Server: nginx/1.25.1
Strict-Transport-Security: max-age=15724800
Vary: Accept-Encoding

2024/05/05 17:51:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 17:51:11 DEBUG : file.ext: Need to transfer - File not found at Destination
2024/05/05 17:51:11 DEBUG : Internet Archive item zorkzorkzork: Waiting for checks to finish
2024/05/05 17:51:11 DEBUG : Internet Archive item zorkzorkzork: Waiting for transfers to finish
2024/05/05 17:51:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 17:51:11 DEBUG : HTTP REQUEST (req 0xc0005106c0)
2024/05/05 17:51:11 DEBUG : PUT /zorkzorkzork%2Ffile.ext HTTP/1.1
Host: s3.us.archive.org
User-Agent: rclone/v1.66.0
Content-Length: 4
Authorization: XXXX
X-Amz-Auto-Make-Bucket: 1
X-Amz-Filemeta-Atime: 2024-05-05T12:27:56.4763362-04:00
X-Amz-Filemeta-Btime: 2023-07-29T12:49:23.4718555-04:00
X-Amz-Filemeta-Mode: 666
X-Amz-Filemeta-Rclone-Mtime: 1970-01-01T00:00:00-05:00
X-Amz-Filemeta-Rclone-Update-Track: loviqis0bezeqij0keyusuq6boqegit6
X-Archive-Auto-Make-Bucket: 1
X-Archive-Cascade-Delete: 1
X-Archive-Keep-Old-Version: 0
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
X-Archive-Queue-Derive: 0
X-Archive-Size-Hint: 4
Accept-Encoding: gzip

2024/05/05 17:51:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 17:51:30 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 17:51:30 DEBUG : HTTP RESPONSE (req 0xc0005106c0)
2024/05/05 17:51:30 DEBUG : HTTP/1.1 200 Ok
Connection: close
Content-Length: 0
Accept-Ranges: bytes
Access-Control-Allow-Headers: authorization,x-amz-acl,x-amz-auto-make-bucket,cache-control,x-requested-with,x-file-name,x-file-size,x-archive-ignore-preexisting-bucket,x-archive-interactive-priority,x-archive-meta-title,x-archive-meta-description,x-archive-meta-language,x-archive-meta-mediatype,x-archive-meta01-subject,x-archive-meta02-subject,x-archive-meta03-subject,x-archive-meta04-subject,x-archive-meta05-subject,x-archive-meta01-collection,x-archive-meta02-collection
Access-Control-Allow-Methods: GET,POST,PUT,DELETE
Access-Control-Allow-Origin: *
Date: Sun, 05 May 2024 21:51:12 GMT
Etag: "9afcb2d16863f2df14342a4143c7e45d"
Server: Apache/2.4.41 (Ubuntu)

2024/05/05 17:51:30 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 17:51:30 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 17:51:30 DEBUG : HTTP REQUEST (req 0xc00099f7a0)
2024/05/05 17:51:30 DEBUG : GET /metadata/zorkzorkzork HTTP/1.1
Host: archive.org
User-Agent: rclone/v1.66.0
Authorization: XXXX
X-Archive-Meta-Creator: It's Me
X-Archive-Meta-Mediatype: audio
X-Archive-Meta-Title: Hey This Works
Accept-Encoding: gzip

2024/05/05 17:51:30 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/05/05 17:51:32 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 17:51:32 DEBUG : HTTP RESPONSE (req 0xc00099f7a0)
2024/05/05 17:51:32 DEBUG : HTTP/2.0 200 OK
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Sun, 05 May 2024 21:51:34 GMT
Onion-Location: https://archive6zg5vrdwm4ljllgxleekeoj43lqayscd4d4kmhnyblq4h3ead.onion/metadata/zorkzorkzork
Referrer-Policy: no-referrer-when-downgrade
Server: nginx/1.25.1
Strict-Transport-Security: max-age=15724800
Vary: Accept-Encoding

2024/05/05 17:51:32 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/05/05 17:51:32 DEBUG : file.ext: Dst hash empty - aborting Src hash check
2024/05/05 17:51:32 INFO  : file.ext: Copied (new)
2024/05/05 17:51:32 INFO  : 
Transferred:   	          4 B / 4 B, 100%, 0 B/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:        24.3s

..please explain me.. i just dont get it.

afaik, need to specify a bucket.

in your first topic about this, i tried to explain that, shared two documentation links and a working command.

then you decided to create this second duplicate topic.
and so, i shared another working command.

thank you vm, i didnt understood that far :slight_smile:

so the right command looks like :

rclone command :slight_smile:

 rclone sync -P /source/path archive:/mydestdirectory/ --metadata --header "X-Archive-Meta-Mediatype: audio" --header "X-Archive-Meta-title: Hey This Works" --header "X-Archive-Meta-Creator: It's Me"  --dump headers

rclone.conf sample :slight_smile:

[archive]
type = internetarchive
access_key_id = 1234
secret_access_key = 5678

#where sync could be replaced by copy
pls not it might not work that good with older versions of rclone

hope it will help ppl..

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.