What is the problem you are having with rclone?
Our Rclone command works well for a long time, suddenly today it is stuck when uploading to S3.
However, when I check Rclone logs, I didn't see any useful information. It is just stuck there. 
Run the command 'rclone version' and share the full output of the command.
rclone v1.66.0
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-79-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.1
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
sudo rclone copy --progress "my-machine:/share/eSIM_Share/device/esim_data_upload/" "my-s3:/hongbomiao-bucket/data/raw-data/my_data/" --filter="- **/*" --filter="+ *.log" -vvv
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[my-machine]
type = sftp
shell_type = unix
host = XXX
port = 22
user = XXX
pass = XXX
md5sum_command = md5sum
sha1sum_command = sha1sum
[my-s3]
type = s3
provider = AWS
access_key_id = XXX
secret_access_key = XXX
region = us-west-2
storage_class = STANDARD
A log from the command that you were trying to run with the -vv flag
...
2024/05/17 17:05:17 DEBUG : my-24_05_16-15_54_14.log: Unchanged skipping
2024/05/17 17:05:17 DEBUG : my-24_05_17-09_18_14.log: Size and modification time the same (differ by 0s, within tolerance 1s)
2024/05/17 17:05:17 DEBUG : my-24_05_17-09_18_14.log: Unchanged skipping
2024/05/17 17:05:17 DEBUG : S3 bucket hongbomiao-bucket path data/raw-data/lab_data: Waiting for transfers to finish
2024/05/17 17:05:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Remote command result: ce9fca0b0518987ddefb1c0b73e39c29 /share/my_lab/device/lab_data_upload/my-24_05_17-10_54_52.log
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_54_52.log: Parsed hash: ce9fca0b0518987ddefb1c0b73e39c29
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_54_52.log: open chunk writer: started multipart upload: gms.NDTeYOtMSnPD.grSjQheqUJP0qlCNICKgM_8im3RsPlG.j0Ifn9za.Hw24t7pBxAtwWAtFmeVmbKSCE4Yenulip9q1e9782s2vyAweIdOpQP2bPTATcwlzQyoxxx
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_54_52.log: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 4
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_54_52.log: Starting multi-thread copy with 599 chunks of size 5Mi with 4 parallel streams
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_54_52.log: multi-thread copy: chunk 4/599 (15728640-20971520) size 5Mi starting
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_54_52.log: multi-thread copy: chunk 3/599 (10485760-15728640) size 5Mi starting
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_54_52.log: multi-thread copy: chunk 1/599 (0-5242880) size 5Mi starting
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_54_52.log: multi-thread copy: chunk 2/599 (5242880-10485760) size 5Mi starting
2024/05/17 17:05:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: New connection 10.11.163.90:34086->10.10.xxx.xx:22 to "SSH-2.0-OpenSSH_8.0"
2024/05/17 17:05:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: New connection 10.11.163.90:34100->10.10.xxx.xx:22 to "SSH-2.0-OpenSSH_8.0"
2024/05/17 17:05:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Remote command result: 991d01532b5b458c1d706a985bea9a19 /share/my_lab/device/lab_data_upload/my-24_05_17-10_48_09.log
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_48_09.log: Parsed hash: 991d01532b5b458c1d706a985bea9a19
2024/05/17 17:05:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: New connection 10.11.163.90:34102->10.10.xxx.xx:22 to "SSH-2.0-OpenSSH_8.0"
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_48_09.log: open chunk writer: started multipart upload: 0_9AEktxk_kWcHcl0aXBBZLrRee9sZqC2B1oxb5mJ125.Tsh5GI7IfkEAB9zuAJ_K5kLJryz7zmU3HFkTFk1ZhV8ec8M9yLuuHVCaAQrJQEZqtVzQz0wvWHobbEC9xxx
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_48_09.log: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 4
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_48_09.log: Starting multi-thread copy with 599 chunks of size 5Mi with 4 parallel streams
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_48_09.log: multi-thread copy: chunk 4/599 (15728640-20971520) size 5Mi starting
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_48_09.log: multi-thread copy: chunk 1/599 (0-5242880) size 5Mi starting
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_48_09.log: multi-thread copy: chunk 2/599 (5242880-10485760) size 5Mi starting
2024/05/17 17:05:37 DEBUG : my-24_05_17-10_48_09.log: multi-thread copy: chunk 3/599 (10485760-15728640) size 5Mi starting
2024/05/17 17:05:38 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Remote command result: ff0b3fb932d95f4032737f2fdf1dfb30 /share/my_lab/device/lab_data_upload/my-24_05_17-12_09_07.log
2024/05/17 17:05:38 DEBUG : my-24_05_17-12_09_07.log: Parsed hash: ff0b3fb932d95f4032737f2fdf1dfb30
2024/05/17 17:05:38 DEBUG : my-24_05_17-12_09_07.log: open chunk writer: started multipart upload: ECQXMAHs6ajloR7G4yrQpC4dLeu9tabHdXg6Ybw_wkjFsftB.Ft_rYMJGCcNF4ZvYo_ykkMwrj13GRWsTry1Am.vkHAlAcMraxaKEIXx8m.ZUB6YAasY6d52wOvzexxx
2024/05/17 17:05:38 DEBUG : my-24_05_17-12_09_07.log: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 4
2024/05/17 17:05:38 DEBUG : my-24_05_17-12_09_07.log: Starting multi-thread copy with 599 chunks of size 5Mi with 4 parallel streams
2024/05/17 17:05:38 DEBUG : my-24_05_17-12_09_07.log: multi-thread copy: chunk 4/599 (15728640-20971520) size 5Mi starting
2024/05/17 17:05:38 DEBUG : my-24_05_17-12_09_07.log: multi-thread copy: chunk 3/599 (10485760-15728640) size 5Mi starting
2024/05/17 17:05:38 DEBUG : my-24_05_17-12_09_07.log: multi-thread copy: chunk 2/599 (5242880-10485760) size 5Mi starting
2024/05/17 17:05:38 DEBUG : my-24_05_17-12_09_07.log: multi-thread copy: chunk 1/599 (0-5242880) size 5Mi starting
2024/05/17 17:05:38 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: New connection 10.11.163.90:34104->10.10.xxx.xx:22 to "SSH-2.0-OpenSSH_8.0"
2024/05/17 17:06:38 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Not closing 2 unused connections as 7 sessions active
2024/05/17 17:07:38 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Not closing 2 unused connections as 7 sessions active
2024/05/17 17:08:38 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Not closing 2 unused connections as 7 sessions active
2024/05/17 17:09:38 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Not closing 2 unused connections as 7 sessions active
2024/05/17 17:10:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Connection failed, closing: connection lost
2024/05/17 17:10:37 DEBUG : my-24_05_17-10_48_09.log: multi-thread copy: chunk 4/599 failed: multi-thread copy: failed to open source: Open failed: connection lost
2024/05/17 17:10:37 DEBUG : my-24_05_17-10_48_09.log: multi-thread copy: chunk 5/599 (20971520-26214400) size 5Mi starting
2024/05/17 17:10:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Connection failed, closing: connection lost
2024/05/17 17:10:37 DEBUG : my-24_05_17-10_48_09.log: multi-thread copy: chunk 1/599 failed: multi-thread copy: failed to open source: Open failed: connection lost
2024/05/17 17:10:37 DEBUG : my-24_05_17-10_54_52.log: Reopening on read failure after offset 1245184 bytes: retry 1/10: connection lost
2024/05/17 17:10:37 DEBUG : my-24_05_17-10_54_52.log: Reopening on read failure after offset 1081344 bytes: retry 1/10: connection lost
2024/05/17 17:10:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Connection failed, closing: connection lost
2024/05/17 17:10:37 DEBUG : my-24_05_17-10_48_09.log: multi-thread copy: chunk 2/599 failed: multi-thread copy: failed to open source: Open failed: connection lost
2024/05/17 17:10:37 DEBUG : my-24_05_17-10_54_52.log: Reopening on read failure after offset 1310720 bytes: retry 1/10: connection lost
2024/05/17 17:10:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: New connection 10.11.163.90:59428->10.10.xxx.xx:22 to "SSH-2.0-OpenSSH_8.0"
2024/05/17 17:10:37 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: New connection 10.11.163.90:59432->10.10.xxx.xx:22 to "SSH-2.0-OpenSSH_8.0"
2024/05/17 17:10:38 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Connection failed, closing: connection lost
2024/05/17 17:10:38 DEBUG : my-24_05_17-12_09_07.log: multi-thread copy: chunk 3/599 failed: multi-thread copy: failed to open source: Open failed: connection lost
2024/05/17 17:10:38 DEBUG : my-24_05_17-10_54_52.log: Reopening on read failure after offset 1736704 bytes: retry 1/10: connection lost
2024/05/17 17:10:38 DEBUG : my-24_05_17-12_09_07.log: multi-thread copy: chunk 5/599 (20971520-26214400) size 5Mi starting
2024/05/17 17:10:38 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: Connection failed, closing: connection lost
2024/05/17 17:10:38 DEBUG : my-24_05_17-10_48_09.log: Reopening on read failure after offset 1900544 bytes: retry 1/10: connection lost
2024/05/17 17:10:38 DEBUG : my-24_05_17-12_09_07.log: multi-thread copy: chunk 4/599 failed: multi-thread copy: failed to open source: Open failed: connection lost
2024/05/17 17:10:38 DEBUG : sftp://my-lab-NAS@10.10.xxx.xx:22//share/my_lab/device/lab_data_upload/: New connection 10.11.163.90:59444->10.10.xxx.xx:22 to "SSH-2.0-OpenSSH_8.0"
Transferred: 0 B / 8.765 GiB, 0%, 0 B/s, ETA -
Checks: 482 / 482, 100%
Transferred: 0 / 3, 0%
Elapsed time: 6m4.4s
Transferring:
* my-24_05_17-10_48_09.log: 0% /2.922Gi, 0/s, -
* my-24_05_17-10_54_52.log: 0% /2.922Gi, 0/s, -
* my-24_05_17-12_09_07.log: 0% /2.922Gi, 0/s, -