What is the problem you are having with rclone?
In the current run of bisync during the first 2 hours there are almost 500 instances of errors like:
2024/12/23 09:14:53 NOTICE: 瀝歃促沩燍餿侔䉌奟/牵纥涄梏㽡頹㽽ᣇ钩␙㘄蓽駯╉ᕩ黚汶ʟ: Failed to read metadata: operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "https://x3k6.ie.idrivee2-50.com/sync/%E7%80%9D%E6%AD%83%E4%BF%83%E6%B2%A9%E7%87%8D%E9%A4%BF%E4%BE%94%E4%89%8C%E5%A5%9F/%E7%89%B5%E7%BA%A5%E6%B6%84%E6%A2%8F%E3%BD%A1%E9%A0%B9%E3%BD%BD%E1%A3%87%E9%92%A9%19%E3%98%84%E8%93%BD%E9%A7%AF%E2%95%89%E1%95%A9%E9%BB%9A%E6%B1%B6%CA%9F": net/http: HTTP/1.x transport connection broken: malformed MIME header line: X-Amz-Meta-E2iid: 瀝歃促沩燍餿侔䉌奟/牵纥涄梏㽡頹㽽ᣇ钩㘄蓽駯╉ᕩ黚汶ʟ
Then after bisync went on to the stage of marking individual files OK errors like this:
2024/12/23 11:25:07 ERROR : PKMS/omnivore export/CSV test/metadata_420_to_440.json: error reading hash from underlying 瀫恷䋇䒏䝪樝煆熟鷟/㟭揣韍樬博霟霂緦囟/媍ꑊ㒀Ꮬ讦䣕辥邏號/ҭ䁎櫴濋⻑敗餰Ⱊ疐祋西身瑢蓮棰㛅␗ɟ: operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "https://x3k6.ie.idrivee2-50.com/sync/%E7%80%AB%E6%81%B7%E4%8B%87%E4%92%8F%E4%9D%AA%E6%A8%9D%E7%85%86%E7%86%9F%E9%B7%9F/%E3%9F%AD%E6%8F%A3%E9%9F%8D%E6%A8%AC%E5%8D%9A%E9%9C%9F%E9%9C%82%E7%B7%A6%E5%9B%9F/%E5%AA%8D%EA%91%8A%E3%92%80%E1%8F%9C%E8%AE%A6%E4%A3%95%E8%BE%A5%E9%82%8F%E8%99%9F/%D2%AD%E4%81%8E%E6%AB%B4%E6%BF%8B%E2%BB%91%E6%95%97%E9%A4%B0%E2%B0%9A%E7%96%90%E7%A5%8B%E8%A5%BF%E8%BA%AB%E7%91%A2%E8%93%AE%E6%A3%B0%E3%9B%85%17%C9%9F": net/http: HTTP/1.x transport connection broken: malformed MIME header line: X-Amz-Meta-E2iid: 瀫恷䋇䒏䝪樝煆熟鷟/㟭揣韍樬博霟霂緦囟/媍ꑊ㒀Ꮬ讦䣕辥邏號/ҭ䁎櫴濋⻑敗餰Ⱊ疐祋西身瑢蓮棰㛅ɟ
Run the command 'rclone version' and share the full output of the command.
rclone v1.68.2
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-127-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.3
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
IDrive.
The command you were trying to run (eg rclone copy /tmp remote:tmp)
I was previously using a hasher but I thought it may help this issue to drop it which is also in line with steps that have been recommended to troubleshoot other issues: Avoiding Bisync ‘times are equal’ conflicts with newer? .
rclone bisync "idrive-sync-crypt:/" "/home/salty/sync/" --check-access --fast-list --verbose --filter-from=/home/salty/.config/rclone/filters --links --compare size,modtime,checksum --log-file="$temp_log" --error-on-no-transfer --conflict-resolve newer --resilient --recover -vv
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[idrive-sync]
type = s3
provider = IDrive
access_key_id = XXX
secret_access_key = XXX
acl = private
endpoint = x3k6.ie.idrivee2-50.com
bucket_acl = private
no_check_bucket = true
server_side_encryption = aws:kms
[idrive-sync-crypt]
type = crypt
remote = idrive-sync:sync
password = XXX
password2 = XXX
filename_encoding = base32768
[idrive-sync-hasher]
type = hasher
remote = idrive-sync-crypt:
hashes = md5
max_age = 3M
auto_size = 107374182400
A log from the command that you were trying to run with the -vv flag
I have debugging enabled but the log contains a lot file names with personal information. I have a script to obfuscate all file and directory names but it is slow to parse and map every file or directory name to something random, especially in a large log. I can share this if necessary or I could search for anything in particular relevant?