SIGSEGV on Ubuntu 24.04

[Danny]

What is the problem you are having with rclone?

When trying to download a file from S3 rclone shows an error with SIGSEGV

Run the command 'rclone version' and share the full output of the command.

$ rclone version
rclone v1.68.1
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.8.0-41-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.1
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy --no-check-certificate near_cf://near-protocol-public/backups/mainnet/rpc/latest ./

The rclone config contents with secrets removed.

[near_cf]
type = s3
provider = AWS
download_url = https://dcf58hz8pnro2.cloudfront.net/
acl = public-read
server_side_encryption = AES256
region = ca-central-1

A log from the command with the -vv flag

$ rclone copy -vv --no-check-certificate near_cf://near-protocol-public/backups/${chain:?}/${kind:?}/latest ./
2024/10/02 16:52:39 DEBUG : rclone: Version "v1.68.1" starting with parameters ["rclone" "copy" "-vv" "--no-check-certificate" "near_cf://near-protocol-public/backups/mainnet/rpc/latest" "./"]
2024/10/02 16:52:39 DEBUG : Creating backend with remote "near_cf://near-protocol-public/backups/mainnet/rpc/latest"
2024/10/02 16:52:39 DEBUG : Using config file from "/home/ubuntu/.config/rclone/rclone.conf"
2024/10/02 16:52:39 DEBUG : Using anonymous credentials - did you mean to set env_auth=true?
2024/10/02 16:52:39 DEBUG : fs cache: adding new entry for parent of "near_cf://near-protocol-public/backups/mainnet/rpc/latest", "near_cf:near-protocol-public/backups/mainnet/rpc"
2024/10/02 16:52:39 DEBUG : Creating backend with remote "./"
2024/10/02 16:52:39 DEBUG : fs cache: renaming cache item "./" to be canonical "/home/ubuntu"
2024/10/02 16:52:39 DEBUG : latest: Need to transfer - File not found at Destination
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x18f48be]

goroutine 1 [running]:
github.com/rclone/rclone/backend/s3.(*Object).downloadFromURL(0xc00027ddd0, {0x2a660f0, 0x3e4fb80}, {0xc00099c345, 0x1a}, {0xc0001263e0, 0x1, 0x1})
	github.com/rclone/rclone/backend/s3/s3.go:5749 +0xa7e
github.com/rclone/rclone/backend/s3.(*Object).Open(0xc00027ddd0, {0x2a660f0, 0x3e4fb80}, {0xc0001263e0, 0x1, 0x1})
	github.com/rclone/rclone/backend/s3/s3.go:5760 +0x387
github.com/rclone/rclone/fs/operations.(*ReOpen).open(0xc0009a4000)
	github.com/rclone/rclone/fs/operations/reopen.go:147 +0x109
github.com/rclone/rclone/fs/operations.NewReOpen({0x2a660f0, 0x3e4fb80}, {0x2a7dc90, 0xc00027ddd0}, 0xa, {0xc0001263c0, 0x1, 0x1})
	github.com/rclone/rclone/fs/operations/reopen.go:102 +0x50b
github.com/rclone/rclone/fs/operations.Open({0x2a660f0, 0x3e4fb80}, {0x2a7dc90, 0xc00027ddd0}, {0xc0001263c0, 0x1, 0x1})
	github.com/rclone/rclone/fs/operations/reopen.go:124 +0x6b
github.com/rclone/rclone/fs/operations.(*copy).manualCopy(0xc00098c120, {0x2a660f0, 0x3e4fb80})
	github.com/rclone/rclone/fs/operations/copy.go:261 +0x4dc
github.com/rclone/rclone/fs/operations.(*copy).copy(0xc00098c120, {0x2a660f0, 0x3e4fb80})
	github.com/rclone/rclone/fs/operations/copy.go:310 +0x176
github.com/rclone/rclone/fs/operations.Copy({0x2a660f0, 0x3e4fb80}, {0x2a7cdb0, 0xc000898640}, {0x0, 0x0}, {0x7ffc8a6de5cd, 0x6}, {0x2a7dc90, 0xc00027ddd0})
	github.com/rclone/rclone/fs/operations/copy.go:412 +0x493
github.com/rclone/rclone/fs/operations.moveOrCopyFile({0x2a660f0, 0x3e4fb80}, {0x2a7cdb0, 0xc000898640}, {0x2a7dc20, 0xc0007ae588}, {0x7ffc8a6de5cd, 0x6}, {0x7ffc8a6de5cd, 0x6}, ...)
	github.com/rclone/rclone/fs/operations/operations.go:2041 +0xcf1
github.com/rclone/rclone/fs/operations.CopyFile(...)
	github.com/rclone/rclone/fs/operations/copy.go:417
github.com/rclone/rclone/cmd/copy.init.func1.1()
	github.com/rclone/rclone/cmd/copy/copy.go:105 +0x79
github.com/rclone/rclone/cmd.Run(0x1, 0x1, 0x3dea1c0, 0xc0007f5c30)
	github.com/rclone/rclone/cmd/cmd.go:251 +0x125
github.com/rclone/rclone/cmd/copy.init.func1(0x3dea1c0, {0xc000511a00, 0x2, 0x4})
	github.com/rclone/rclone/cmd/copy/copy.go:101 +0xbd
github.com/spf13/cobra.(*Command).execute(0x3dea1c0, {0xc0005119c0, 0x4, 0x4})
	github.com/spf13/cobra@v1.8.1/command.go:989 +0xa91
github.com/spf13/cobra.(*Command).ExecuteC(0x3de4000)
	github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/cobra@v1.8.1/command.go:1041
github.com/rclone/rclone/cmd.Main()
	github.com/rclone/rclone/cmd/cmd.go:535 +0x31
main.main()
	github.com/rclone/rclone/rclone.go:14 +0xf

[ncw (Nick Craig-Wood)]

This is definitely a bug which was caused by the SDKv2 conversion.

Please give this a go

v1.69.0-beta.8356.d7b5aa867.fix-s3-downloadurl-crash on branch fix-s3-downloadurl-crash (uploaded in 15-30 mins)

[Danny]

This works fine now! Thanks. Note that the upstream ubuntu release 1.60 has this bug. shall I report it to ubuntu when we have a fixed build? I assume that will get merged into 1.69 since it's a trivial change?

[ncw (Nick Craig-Wood)]

Thanks for testing. I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.69 and it will go in v1.68.2 if we make one of those.

@Danny

Note that the upstream ubuntu release 1.60 has this bug. shall I report it to ubuntu when we have a fixed build?

The patch I did for 1.68 won't work for 1.60 as we changed the s3 SDK version in 1.68

If 1.60 has a bug I don't think it is exactly this one due to the old SDK taking a pointer where as the new one doesn't which caused the crash when we dereferenced a nil pointer.

[Danny]

Ok great thanks. From memory the stack trace was the same on the upstream package - if I get time I’ll try to make a bug directly with upstream and point them to the releases containing the fix.

[Danny]

Never mind, I checked and the upstream version doesn't have any issue. we can close this off, thanks for the help!