No copy from azure blob after 1.70

Oleksandr_Isniuk · June 24, 2025, 3:50pm

What is the problem you are having with rclone?

After the update to 1.70, the scheduled job stoped working. Debug log provides no additional info. Connection happens through the SAS token. It wasn't expired. It has enough permissions. At least for the previous version.

Rolling back to version 1.69 fully resolves the issue. Version is the only thing that was changed. Same vars/sas/config/command/etc.

Run the command 'rclone version' and share the full output of the command.

rclone v1.70.1

os/version: ubuntu 24.04 (64 bit)
os/kernel: 5.15.0-1090-azure (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.24.4
go/linking: static
go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Azure blob

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone config create crypt crypt directory_name_encryption false \
    filename_encryption off password $(cat rnsalt) password2 $(cat rnpass) remote s3:$S3_BUCKET && \
    rclone -vv --s3-no-check-bucket --s3-no-head --max-backlog 1000 --size-only --azureblob-disable-checksum \
    --azureblob-sas-url "https://${AZ_ACCOUNT}.blob.core.windows.net/${AZ_CONTAINER}?$(cat /home/ubuntu/az/sas_token)" --azureblob-no-head-object \
    --no-traverse copy az:$AZ_CONTAINER crypt:$DATE_MARK/ --files-from-raw blob-list.txt

The rclone config contents with secrets removed.

[az]
type = azureblob
account = sa_acc_name
tenant = ...

[s3]
type = s3
provider = AWS
env_auth = true
region = eu-north-1

[crypt]
type = crypt
password = ...
password2 = ...
remote = s3:bucket_name
directory_name_encryption = false
filename_encryption = off

A log from the command with the `-vv` flag

2025/06/24 11:13:00 DEBUG : rclone: Version "v1.70.1" starting with parameters ["rclone" "-vv" "--s3-no-check-bucket" "--s3-no-head" "--max-backlog" "1000" "--size-only" "--azureblob-disable-checksum" "--azureblob-sas-url" "https://sa_acc_name.blob.core.windows.net/container_name?fully_working_sas_token_here" "--azureblob-no-head-object" "--no-traverse" "copy" "az:container_name" "crypt:dir_name/" "--files-from-raw" "blob-list.txt"]
2025/06/24 11:13:00 DEBUG : Creating backend with remote "az:container_name"
2025/06/24 11:13:00 DEBUG : Using config file from "/home/ubuntu/.config/rclone/rclone.conf"
2025/06/24 11:13:00 DEBUG : az: detected overridden config - adding "{...}" suffix to name
2025/06/24 11:13:00 DEBUG : fs cache: renaming cache item "az:container_name" to be canonical "az{...}:container_name"
2025/06/24 11:13:00 DEBUG : Creating backend with remote "crypt:dir_name/"
2025/06/24 11:13:00 DEBUG : Creating backend with remote "s3:bucket_name/dir_name/.bin"
2025/06/24 11:13:00 DEBUG : s3: detected overridden config - adding "{---}" suffix to name
2025/06/24 11:13:00 DEBUG : fs cache: renaming cache item "s3:bucket_name/dir_name/.bin" to be canonical "s3{---}:bucket_name/dir_name/.bin"
2025/06/24 11:13:00 DEBUG : Creating backend with remote "s3:bucket_name/dir_name"
2025/06/24 11:13:00 DEBUG : s3: detected overridden config - adding "{---}" suffix to name
2025/06/24 11:13:00 DEBUG : fs cache: renaming cache item "s3:bucket_name/dir_name" to be canonical "s3{---}:bucket_name/dir_name"
2025/06/24 11:14:00 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 0 / 0, -, Listed 29066
Elapsed time:       1m0.0s

2025/06/24 11:15:00 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 0 / 0, -, Listed 29066
Elapsed time:       2m0.0s

2025/06/24 11:16:00 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 0 / 0, -, Listed 29066
Elapsed time:       3m0.0s

2025/06/24 11:17:00 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:                 0 / 0, -, Listed 29066
Elapsed time:       4m0.0s

ncw · July 3, 2025, 9:26am

This is very likely this problem

github.com/rclone/rclone

deadlock in --no-traverse when all transfers skipped

opened 09:23AM - 02 Jul 25 UTC

ncw

bug

Issue made from https://github.com/rclone/rclone/pull/8647 by @jeremy #### Wha…t is the problem you are having with rclone? When every source object already exists at the destination and `--no-traverse` is set, march stalled/deadlocked after 100 entries because dstChan lagged srcChan. Introduced in v1.70.0 with callback-based syncing. I couldn't get a test case working reliably, but here's a script to reproduce: ```bash #!/usr/bin/env bash # Reproduces rclone v1.70+ deadlock with --no-traverse when all files exist # Usage: ./repro.sh [rclone-binary-path] # Use provided rclone binary or default to system rclone RCLONE="${1:-rclone}" echo "Using rclone: $RCLONE" echo "Version: $($RCLONE version | head -1)" echo # Create test directories src=$(mktemp -d) dst=$(mktemp -d) # Create 200 test files (enough to trigger the ~100 file deadlock) echo "Creating 200 test files..." for i in {1..200}; do echo "content$i" > "$src/file$i.txt" done # Initial copy to establish destination echo "Initial copy to destination..." $RCLONE copy "$src/" "$dst/" --quiet # Create file list filelist=$(mktemp) ls "$src" > "$filelist" # This command deadlocks in v1.70+ when all files exist at destination echo "Testing --no-traverse with all files existing (10s timeout)..." timeout 10s $RCLONE copy "$src/" "$dst/" \ --no-traverse \ --files-from "$filelist" \ --ignore-existing \ --progress if [ $? -eq 124 ]; then echo -e "\n❌ FAIL: Command timed out (deadlock detected)" exit 1 else echo -e "\n✅ PASS: Command completed successfully" fi # Cleanup rm -rf "$src" "$dst" "$filelist" ``` Results on 1.69.3, 1.70.2, and with this patch: ```bash Using rclone: rclone Version: rclone v1.69.3 Creating 200 test files... Initial copy to destination... Testing --no-traverse with all files existing (10s timeout)... Transferred: 0 B / 0 B, -, 0 B/s, ETA - Checks: 200 / 200, 100% Elapsed time: 0.0s ✅ PASS: Command completed successfully ``` ```bash Using rclone: rclone Version: rclone v1.70.2 Creating 200 test files... Initial copy to destination... Testing --no-traverse with all files existing (10s timeout)... Transferred: 0 B / 0 B, -, 0 B/s, ETA - Checks: 0 / 0, -, Listed 200 Elapsed time: 9.5s ❌ FAIL: Command timed out (deadlock detected) ``` ```bash Using rclone: ./rclone Version: rclone v1.71.0-DEV Creating 200 test files... Initial copy to destination... Testing --no-traverse with all files existing (10s timeout)... Transferred: 0 B / 0 B, -, 0 B/s, ETA - Checks: 200 / 200, 100%, Listed 200 Elapsed time: 0.0s ✅ PASS: Command completed successfully ``` #### What is your rclone version (output from `rclone version`) v1.70.2 #### The associated forum post URL from `https://forum.rclone.org` * https://forum.rclone.org/t/1-70-1-appears-to-hang-when-using-files-from-rclone-ls-no-traverse-and-ignore-existing/51720 * https://forum.rclone.org/t/deadlock-when-using-no-traverse-max-duration-and-files-from-with-many-files-100/51650 #### How to use GitHub * Please use the 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to show that you are affected by the same issue. * Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue. * Subscribe to receive notifications on status change and new comments.

There is a binary there to try to see if it resolves the problem